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The manual provides vocational educators with 
information and guidelines regarding the design and use of 
questionnaires and interviews for gathering data and the 
construction, tabulation, and analysis of both open ended and forced 
response survey instruments. The first of nine chapters deals with 
surveys as a data collection technique, discusses the use of printed 
instruments, other information collecting techniques, and additional 
guidelines to determine the selection of survey type. Chapter 2, 
writing questions, covers plans for question building and response 
modes. Chapter 3^ forced response questions, deals with establishing 
appropriate scales and includes self-- practice itezas. Chapter 4, 
coding survey items, discusses forced response and open response 
codes* Chapter 5, preventing biased results, deals with wording the 
questions, sampling considerations, and selecting the appropriate 
response type. Chapter 6, conducting the survey, discusses knowing 
the target population, protecting the participants, using 
inclusionary language, and communicating with the target population. 
Chapter 7, tabulation of data, discusses methods of tabulation, 
especially electronic tabulation. Chapter 8, statistical analysis of 
data, discusses descriptive techniques and the use of statistical 
tests. The last chapter, the research report, covers planning and 
constructing the report. An appendix includes three model 
questionnaire formats. The document is indexed. (JB) 
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PROLOGUE 

You have just completed a very, very busy Friday at 
school (hang in there . . . vacation starts next week). As 
you plow through the stack of mail on your desk, you spot a 
large, suspicious-looking envelope. You open it, and find 
there, in all its glory, a dittoed (very faint blue print, 
of course), 12-page, single spaced, questionnaire. The 
writer humbly requests that you complete and return it within 
three days. There is a poorly written cover letter and 
no self -addressed return envelope. 

With your schedule being as jammed as it is, and a 
feeling of professional duty surging through your heart, you 
vow that the questionnaire shall be returned, posthaste. 
Being a prudent manager of time, you ask a student helper to 
complete the questionnaire for you. Rationale: the student 
helper can answer as clearly as you, since the questionnaire 
is confused, disorganized, and ambiguous. 

Let's hear it for the Big Q! 

If the above example approaches any reality, you'll 
appreciate an effort to help stamp out poorly designed 
questionnaires. That is where this book begins! 
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PURPOSE OF MANUAL 



The purpose of this manual is to provide vocational 
educators with information which can assist them in designing 
questionnaires, or as they are often called — data gathering 
instruments* Data obtained through a survey can enhance a 
vocational educator's ability to plan new occupational 
programs, revise or improve current programs, or delete 
obsolete programs. 

The manual contains a discussion of various uses and 
abuses regarding the use of questionnaires and interviews 
for gathering data* In addition, models are included that 
illustrate how to construct, tabulate, and analyze both 
open ended and forced response instruments*^ 

In Washington, during 1974, approximately twenty 
"Skills Centers Feasibility Studies" were completed which 
used questionnaires of varying quality and design. Articles 
published in professional journals (e.g.. School Shop , The 
Journal of Home Economics ) and other funded projects revea"* 
that the questionnaire is a most commonly used tool to deter- 
mine the needs of vocational education programs. It is 
important that data are collected in a scientifically 
respectable manner. The conclusions drawn from the data 

then will be useful and stand the test of critical review. 
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Guidelines are presented in this manual to design 

questionnaires by those individuals who may not have access 

to specialists. The writers assume that this manual will be 

valuable on an independent study basis and provide: 

1* A checklist for the experienced educator or novice 
to evaluate the design of the questionnaire as to 
bias, objectivity, and workability. 

2. Information on designing questions. 

3. Suggestions about tabulati^g^/and., analyzing the data 
which are gathered. ^ . ^ ^ > 

4. Suggestions about methods for communicating and 
illustrating data. 

Methods for selecting sample populations, follow-up techniques, 

budget, and time limitations are included in the manual. 

To further assist the occupational and career educator 

in conducting a successful investigation, a suggested sequence 

of events is provided. Each event is discussed and expanded 

in specific chapters. The sequence of events is shown here 

to provide a quick reference check to determine "Where am I?" 

in the data gathering process. 

Sequence of^Events 

1. Determine the purpose of the study. 

2. Determine the target population. 

3. Determine the method of investigation. 

4. Determine question foinnat. 

5. Write and code answers. 

6. Plan method of tabulation. 



7. 


Plan method of analysis. 


8. 


Determine the sample population. 


9. 


Write cover letter. 


10. 


Pretest the instrument. 


11. 


Conduct the survey. 


12. 


Account for returns. 


13. 


Follow up for non-'respondents . 


14 . 


Process returned data. 


15. 


Analyze returned data. 


16. 


Compile research findings. 


17. 


Display findings. 


18. 


Write final report. 




The manual is designed to address each of the above 



events in a direct and explanatory manner. The experienced 
educator and novice alike will benefit from its step-by-step 
approach. The express purpose of the manual is to improve 
the quality of data collected in the schools. 
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CHAPTER 1 

SURVEYS AS A DATA COLLECTION TECHNIQUE 

A decision to conduct a survey should be made only 
after serious consideration to the alternatives. In far too 
many cases, researchers conduct a survey via a questionnaire 
and obtain data that could have been abstracted from primary 
source reports or state forms. The mailing of questionnaires 
is most widespread. A district superintendent of schools 
told two of the writers of this manual that he completes an 
average of one questionnaire per day! If these data gathering 
techniques are used that widely, is it any wonder why so few 
returns approach 100 percent? 

Thus, the first decision to make when designing a 
survey is to determine if the data are available in already 
collected sources. If the response is in the negative, the 
next decision should address the questions of, "Is this survey 
really needed?" "Will the information affect some policy?" 
Finally, one must ask, "What is the problem or needs area on 
which the survey will focus?" When these decisions have been 
made you are ready to plan for the conduct of a survey. 



The focus . All surveys focus on a selected problem, 
need, or condition. The first task in conducting a survey 
should be to determine what has already been written or 
published about the problem. This step is often called a 
"Review of the Literature." By conducting a thorough review 
of published or unpublished reports, comparisons can be made 
between respondent groups. Surveys are often accomplished 
to determine the state of affairs of some trait. Usually 
questionnaires are data collection systems for so called 
"status" studies. One conducts a survey to determine, for 
example, the status of vocational tailoring, shop equipment, 
or career counseling. 

The important point that must be stressed is that 
surveys always focus on some specific condition, state, need, 
or problem. A survey is not prepared from a template as one 
prepares cutting boards for mass sale. A survey is always 
unique to the situation. With this major point being 
emphasized, let us discuss the use of a printed questionnaire 
in data collection. 

Using Printed Instruments 

The Questionnaire; Advantages 
and Disadvantages 

Each questionnaire must be constructed to address well 

established criteria or specific objectives. IVhen designed 

and used in this manner, questionnaires can be an efficient 
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means by which to gather data. Many educational researchers 
have commented on the advantages and disadvantages of the 
survey questionnaire. By examining Sax (1968), Tuckman 
(1972), Clock (1967), and Oppenheim (1966), we have generated 
a list of common advantages and disadvantages of using 
questionnaires . 

General advantages . The following list summarizes 
the overall general advantages of using a questionnaire to 
conduct a survey. 

1. Many individuals may be contacted at the same 
time, usually through the mail. 

2. A questionnaire is less expensive than the interview 
technique. 

3. Each selected respondent receives identical 
questions. 

4. A written questionnaire provides a vehicle for 
expression without fear of' embarrassment to the 
respondent. 

5. Responses are easily tabulated (depending on design 
of instrument) . 

6. People may answer at their own convenience. 

7. There is no need to select and train interviewers. 

8. People in remote or distant areas are reached. 

The above eight advantages must be carefully analyzed 

for each specific survey . Further, the timing of a survey 

is critical. Some school districts have policies which 

specify that no one is obliged to complete any questionnaire 

which has not been approved by the district. Additionally, 

the months of September, December, January, May, and June of 
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the school year are very poor times in which to mail a 
questionnaire. The sximmer months of July and August are out. 
Thus, a researcher should attempt to contact the intended 
respondents in October, November, February, March, and April. 
These months tend to be less busy in completing reports by 
school personnel. 

Timing is also important to the researcher. Often 
deadlines are being met and reports or recommendations based 
on the survey are promised to decision makers. Therefore, 
plans must be made far in advance of any actual submission of 
a questionnaire. 

As one example, two of the writers of this guide 
conducted a rather short survey of vocational agricultural 
teachers in Washington. The instrument had only sixteen items. 
Yet, it took over six different drafts of the instrument and 
at least two months before all items were precisely stated 
and all focusing on the general problem area. What took so 
long? Each draft was critiqued by persons not familiar with 
the survey. As criticisms arrived, the items were modified, 
deleted, or replaced. This process took time, but it had been 
planned with the time lag in advance. 

All persons who conduct reputable surveys must take 
great care in item selection to eliminate any possible chance 
for unintended bias. This topic is discussed in Chapter 3 at 
great length. The general advantage of a questionnaire survey 
must be applicable to the survey if it is to be successfully 



completed. Now let us focus on the opposing end of the 
spectrum. 

Disadvantages of a questionnaire . For every advantage 
to Conducting a questionnaire survey there is an accompanying 
disadvantage. A summary of disadvantages yields the following 
list: 

1. The investigator is prevented from learning the 
respondent's motivation for answering questions. 

2. Respondents may be limited from providing free 
expression of opinions. 

3. The collection of data from individuals who cannot 
read^ write ^ or see is prevented. 

4. The return of all questionnaires is difficult to 
achieve. 

5. Complex designs cause poor responses or none. 

6. The investigator is prevented from learning what 
causes a problem. 

7. The name and current addresses of the target 
population are often not available. 

8. A question may have different meanings to different 
people. 

9. There is no assurance that the respondent actually 
completes the instrument. 

10. Selections of the sample^ per se^ may cause biased 
results. 

11. The questionnaire asks for long outdated inf oin[nation. 

12. Respondents may not complete the entire instrument. 
The above list may be far from complete but an 

investigator must realize the basic limitations of a written 
questionnaire. If the limitations do not render the survey 
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void/ then a questionnaire may be used. There are other 
concerns. For example, two of the writers conducted a state- 
wide survey, unknowingly, at the same time that a similar 
survey was being circulated to the same target group. 
Undoubtedly, these two simultaneous surveys adversely affected 
the respective returns. We were a bit amused and chagrined 
to find that someone else's questionnaire had even been 
returned in our self-addressed, stamped envelope! (By the 
way, we did get a 97% return.) 

But, if a written questionnaire is not used what are 
the alternatives? There are two: personal interviews and 
telephone interviews. Let us focus our attention on these 
techniques. 

Other Information Collecting Techniques 

The Interview; Advantages and 
Disadvantages 

Several researchers, including those previously cited, 
have discovered some common characteristics about personal 
interviews. One of the most favorable aspects is the 
interviewer's personal contact with the respondents. In this 
setting, one can clarify any question immediately. A 
generalized set of advantages of the interview includes the 
following items : 

1. Feelings of the respondents are revealed. 

2, Discussion about the causes of problems or solutions 
to problems is allowed • 



3. The respondent is allowed maximum opportunity for 
free expression. 

4. The interviewer can observe non-verbal behaviors. 

5. Respondents may provide personal information, 
attitudes, beliefs, and perceptions that might not 
be gained on a written instrument. 

6. A high rate of participation is provided. 

7. The interviewer has an opportunity to follow-up 
or probe leads. 

8. Individuals who cannot read, write, or see can 
participate. 

9. Fewer individuals may be needed than for mailed 
surveys. 

A survey conducted by interview would take much longer 
to complete than a questionnaire survey, but the informants 
(respondents) tend to comply. One major consideration is the 
recording of information obtained during an interview. An 
interviewer may use a tape recorder, or write notes as the 
interview takes place. Some infomants will not allow an 
interviewer to tape record while others get neirvous about 
written notes taken during the conversation. This is one 
aspect of an interview that cannot be predicted in advance. 
The interviewer should ask the informant if the interview may 
be tape recorded or if taking notes would be bothersome. The 
interviewer might even tell the infojrman": that the responses 
which are recorded will be reviewed by the informant for 
editing or other comments. 

If the size of the sample group is small, perhaps 
under twenty-five, the interview technique would be 
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appropriate. This may not be so for telephone interviews 
which will be discussed later in the chapter. But, if the 
twenty-five are scattered throughout a region or state the 
cost to conduct a personal interview would be exorbitant. 

Interviews should be used mainly to determine 
attitudes or opinions. Yet discretion must be exercised lest 
sensitive questions be introduced which could alienate the 
informant. 

Interview types . As with questionnaires there are 
two general types of interviews: directed and focused. The 
directed interview is highly structured. An interview 
schedule or guide is prepared in advance. A focused or 
unstructured interview is not as completely structured with 
a well prepared schedule as is the directed type. The 
interviewer has some basic topic identified and then probes 
into areas which seem fruitful to explore. A structured 
interview does permit a similar set of questions to be 
answered by all; while the unstructured interview type may 
lead to problems of arriving at a consensus or set of general- 
izations. 

Conducting an interview . Regardless of the type of 
interview used, some plan is required. Generally, an 
interviewer begins with a series of structured and perhaps 
fact questions. These are usually non-threatening questions 
and allow the informant to "loosen up." After the initial 
items, more pressing or feeling-types of questions can be asked. 

8 



If more than one person conducts an interview^ then 
detailed training sessions must be conducted. All interviewers 
must be trained to avoid interjecting bias in the questions 
through the verbal aspects of the interview or non-verbal 
reactions. Even if one person does the interviewing it is 
essential to practice several trial interviews. If possible^ 
video tape the trials to determine if the interviewer sessions 
were non-biased. 

It is essential that all interviews be scheduled via 
an appointment with the informant. In most cases ^ the 
informant's schedule will take precedence over the inter- 
viewer's. When appointments are made/ explain the objective 
of the interview and then set an appointment. Be prepared to 
spend a greater amount of time with each informant than you 
might have planned. As with questionnaires ^ cover letters 
explain why you are there and that all responses will be 
treated confidentially. These same points must be verbalized 
in an interview. 

Disadvantages . The interview is usually more 
expensive than a mailed survey for three reasons: training , 
transportation / and wages. Major disadvantages of the 
interview technique include the following points: 

1. The method is time consuming. 

2. Only a limited number of persons may be interviewed 
due to time and cost. 

3. Quantification of results may be difficult. 

4. Scheduling of interviews may be difficult. 
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5. Cost may be prohibitive. 

6. Respondents might feel that they are being "put 
on the spot. " 

7. The interviewer may make subjective judgments about 
the responses. 

8. The overall reliabilit^y of responses can be limited 
since respondents tend to answer truthfully those 
questions which are not embarrassing to them. 

9. Interview responses are sometimes biased depending 
upon the age, sex, education, race, interview 
experience, socio-economic level, and religious 
background of the interviewer. 

The criteria for any survey will in part play a 
deciding role in the determination of conducting an interview. 
If the advantages of an interview are greater than those of 
a mailed questionnaire, the interview technique will be very 
revealing. 

Telephone interviews . The telephone interview is also 
a method available to investigators. The same general 
constraints and rules apply to telephone interviews as with 
the personal interview. However, in telephone interviews the 
structured type of schedule may be preferable to the 
unstructured since people tend to consider telephone 
conversations as somewhat privileged. 

Telephone interviews may be conducted via long distance 
which reduces travel costs. Interviev/s conducted over the 
telephone will be highly reliable if the interview schedule 
is non-biased. Persons respond freely to telephone surveys 
once initial rapport and trust have been established. 
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Telephone interviewing is a very effective method^ especially, 
when seeking opinions on issues relating to public policy. 
Of course, a structured interview is easier to conduct and 
less expensive than is an unstructured one. The schedule which 
follows provides some ideas on how to proceed in the conduct 
of any interview. 

The interview schedule . The plan or outline to be 
used when conducting the interview is the "schedule." As 
stated before/ the schedule assists the interviewer to remain 
on track and helps to ensure that all informants have an 
opportunity to respond to the identical phrasing of a partic- 
ular question. 

A main point to remember when designing the interview 
schedule is to avoid questions that can be answered with a 
"Yes" or "No" response. One purpose of an interview is to 
gather details that are unique in the perception of the 
informant. Also, avoid language that might be considered 
offensive by the respondent. 

One type of question that the interview schedule might 
contain is one that will provide some fact data such as: 
"How long have you lived in this city?" 

After a few of these, lead into more open ended 
questions that provide subjective data from the informant 
such as: 

"To what extent do you think that the present vocational 
education program at the high school is meeting the 
needs of our youth?" 
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The interviewer must decide whether to record responses 
during the interview or immediately after the interview. It 
is recommended that the interviewer concentrate on what the 
respondent is actually saying. This takes a great deal of 
practice if accuracy is to be maintained. If delay in writing 
responses is used^ they must be written immediately after the 
interview. The format which follows could be used when 
constructing an unstructured interview schedule. 

A Model of an Interview Schedule 
PARENT INTERVIEW 



Name Date 

Address Time 



1. How long have you lived in this city? 

2. Did you attend high school here? 

3. How many children attend high school in this city? 

4. What type of vocational education courses should be 
provided for high school students? 

5. What type of vocational education courses should not be 
provided for high school students? 

6. What changes would you like to see in the present high 
school vocational education program? 

7. Describe the type of vocational skills you feel should 
be taught ia a high school vocational education program., 
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Observe in the above schedule that items 4-7 could 
be interchanged by the interviewer • In a structured interview^ 
the interviewer would proceed through a definite list of 
items ^ just as one completes a questionnaire. When using an 
unstructured technique the interviewer may probe for additional 
comments or may alter the sequences of the planned set of 
questions. Both techniques require training and should be 
used only with such training for the most effective conduct 
of the interview. 

Additional Guidelines to 
Determine Surveys 

Another important planning consideration for the type 

of survey^ either questionnaire or interview^ is the budget 

limitation. It is crucial to project realistic cost analyses 

for the following tasks or elements: 

1. Printing cost of the questionnaire, cover letter, 
or special forms. 

2. Postage for initial mailing and follow-ups. 

3. Typing of original and revised questionnaires 
after field testing. 

4. Editing, marking, and tabulating returned 
questionnaires . 

5. Electronic data processing needs, if required 
(including key punch and machine time). 

6. Office supplies to be used. 

7. Additional or part-time staff wages. 

8. Typing of the final report. 
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Costs of mailed surveys are similar to those of the 
personal interview, except the postage cost will be replaced 
by interviewer cost, including the training of the interviewer, 
and the actual cost for the field test of the interview 
instrument. To aid in planning the costs of a survey, a 
somewhat realistic example is presented below. 

Sample Survey Budget 

The following budget figures are for illustration 
purposes only. Actual cost figures may vary a great deal 
from those figures shown below due to fluctuating economic 
conditions. 

Wages . Plan to spend no less than $3.00 per hour for 
clerical help, it may be prudent to retain part-time staff 
persons (four hours a day) for at least four work weeks, at 
a total cost of $24 0.00. This item may be excluded from your 
budget by investigating the use of educational aides already 
employed by your school. In addition, you might request 
assistance from business occupation students through a course 
instructor. Consultant fees vary from $35.00 a day to over 
$200.00 a day. If consultants are required, be certain to 
plan for their costs. 

Services . There are many variables associated with 
electronic data processing. This budget item can be determined 
by asking computer specialists for cost estimates. If you 
have about 4 00 respondents, and a rather uncomplicated survey, 
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a $125 ♦ 00 budget would probably be adequate. But, the more 
complex the analyses, the greater the cost. 

Materials . Cost of materials varies, depending on 
the source of supplies. Paper and office supplies can run 
about $50. Postage approaches $200 for approximately 
400 respondents. Although the estimated budget will vary 
with each survey, typically it costs about 50 cents to $1.00 
per respondent for the mailing of a questionnaire which 
includes a self -addressed envelope and two follow-ups. 
Remaining postage can be used for the first follow-up, 
usually a postcard. A second follow-up, if required (and it 
often is) , would include a letter and possibly another 
questionnaire. 

The budget for a survey can easily approach $1,000 
when one must pay for part-time help to complete the job. 

This chapter has presented some general aspects that 
are important to decisions for conducting surveys. Once a 
decision is made to conduct a questionnaire survey, then, the 
writing of the questions is essential. That topic will be 
addressed in Chapter 2. 
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CHAPTER 2 



WRITING QUESTIONS 
Plans for Question Building 

Question Content 

Writing appropriate and carefully formulated questions 
for the instriiment may be the single most time consuming and 
important job of conducting a survey. The content of each 
question determines whether or not the survey will help 
elicit the desired information. When writing questions, many 
criteria must be considered; Marie Jahoda, et al. (1951), 
compiled a comprehensive list of such criteria. Accompanied 
with examples and explanations, the list, with modifications, 
follows. 

Is the question necessary? Ask only for information 
that is needed for the present study. Questions that would 
be nice to know or can be used for another study clutter up 
a questionnaire and often threaten the credibility of the 
study. Selecting appropriate questions is dependent on having 
precisely formulated objectives for the survey. Plan how the 
information f^-om each question will be used. 

Surveys taken in schools, for example, frequently ask 
a number of objective questions such as respondent's age, 
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sex, parents' occupation, major in school^ address^ "for 
possible use at some later date." If the information is 
not needed for the intended study, do not request it. Sensi- 
tive questions, such as respondent's income, should definitely 
be avoided. 

It may be possible to ask a related question to 
obtain the desired information. For example, if it is 
essential to know the respondent's income, ask for the 
occupation and then approximate the income by checking local 
occupational salary averages available from the U.S. Bureau 
of the Census or local Employment Security Offices. 

When seemingly unnecessary or sensitive questions 
are included in a survey, a typical participant reaction is 
to "throw the whole thing away." Only one or two inappropri- 
ate questions can destroy credibility and therefore hinder 
response rate. 

Are several related questions needed ? Questions 

should not be too general or too detailed. If more than one 

aspect of a topic is addressed in a single question, the 

question will be ambiguous. The following exemplify survey 

questions which need improvement. The discussions which 

follow illustrate how the questions can be improved. 

1. Do you' like shop and physical education? 

( ) Yes 

( ) No 

( ) Neutral 

Two distinct subjects are addressed in this question. 

If only one activity is enjoyed, the respondent's opinion 
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will not be reflected by any of the response categories. 
To obtain valid data, each of these subjects must be con- 
sidered in a separate question, 

2. Are you in favor of vocational education? 

( ) Yes 
( ) No 

( ) Do not know 

This question is too vague* The word "favor" will 
be subjectively defined by each respondent. A more specific 
question would produce usable data, e.g., "What percentage 
of the students in your high school should be provided with 
vocational training? (1) 0-25, (2) 26-50, (3) 51-75, 
(4) 76-100." 

3. Should the school offer a class on "How to find a 
job? " 

( ) Yes 
( ) No 

( ) Do not know 

Respondent may answer "yes" and have no intention of 
enrolling in such a course. If the purpose of the question 
is to assess the need for the course, ask if the respondent 
would enroll in the class. 

4. What is your favorite subject? ^ 



The results from this question may indicate little 
or no interest in vocational education, hov7ever, the results 
may not be conclusive. A more reasonable approach would be 
to have the respondent rank a series of various clusters of 
subjects. Yet, favorite subjects may not be the subject that 
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a student values most. Words must be carefully selected 
according to the purpose of the study, to avoid biasing the 
results. 

5. What vocational courses would you like added to 
our list of offerings? 

In this case, more additional information should be 

provided, such as the current listing of vocational courses. 

In that manner the question can be more precisely asked, 

with attitudes associated with the question being assessed. 

Question 5 should be accompanied with another question that 

would ask if the student plans to attend college, since 

college bound students may respond with a "none." Further, 

students entering science or engineering related courses in 

universities might find industrial arts or vocational offerings 

relevant. Thus, a series of questions might be developed 

which would focus specifically on their opinions. 

6. How many hours do you usually work each week? 

A seasonal worker, such as some construction workers, 
may respond to this question "60 hours per week." To get an 
accurate calculation of actual working time, a question must 
also be asked that will reveal the length of the usual layoff 
periods. If this question were asked of high school students 
the results would be chaotic since the question is too vague. 

7. Do you think there is a need to build an addition 
to the business-education wing? 

( ) Yes 
( ) No 

( ) No opinion 
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In addition to seeking opinions^ it may be asked if 
the responc'ent is willing to establish an alternative for 
such a facility. Try not to fall into the "either-or" 
trap* Such questions do not provide adequate data for 
reasonable decisions^ especially when alternatives are being 
sought. Lastly^ the question may be very poor if the respon- 
dents were unaware of current building and classroom utili- 
zation. 

In the preceding examples^ a response continuum may 

be substituted for each ( ) Yes, ( ) No, ( ) No opinion 

question. For example, "There is a need to build an addition 

to the business-education wing." 

strongly agree 

agree 

no opinion 

disagree 

strongly disagree 

The above illustrates how a continuum with degrees of 

approach or avoidance may be constructed. 

Criteria for selecting the most appropriate response 

categories are discussed in Chapter 6. 

Do respondents have the information necessary to 

answer the question ? Occasionally questions are asked to 

which individuals cannot respond. Allow for this situation 

by providing non-content choices such as "Do not know" or 

"No opinion." However, if the non-content responses are 

marked frequently, the wrong group has probably been surveyed 

^or the wrong questions have been asked. For example, it 
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would be of little value to poll public opinion concerning 
the adequacy of a school's insurance coverage for shop 
courses* Probably most people have no knowledge that such 
a policy even exists. 

Consideration must also be given to the respondent's 
interest in the topic* If the general interest is minimal/ 
completed returns will be practically nil* One means by 
which to increase interest in a questionnaire is to ask 
questions designed specifically for the subgroup. A section 
in a questionnaire may be directed toward a specific subgroup 
of respondents (e.g., male or female, vocational instructors 
or counselors, teachers or administrators) . If the subgroups 
are thought to be very heterogeneous, separate questionnaires 
might be constructed for each selected group. 

For example, to evaluate the success of certain 
vocational courses which might have been recently offered, 
instructors may be polled to assess student success in 
meeting class objectives. In this case, student opinions 
about the courses should also be polled. When planning a 
survey, plan to question all concerned groups so that the 
various studies will be complementary to each other. 

Improving the items . Below are some specific 
examples of questions which could be asked in a survey. 
Immediately following each example are some suggestions for 
improving their construction. 
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8. Do you plan to enroll in a vocational school after 
high school graduation? 

( ) Yes 
( ) No 

( ) Do not know 

This type of question may be used to separate 
respondents into subgroups. Those respondents who answered 
positively may then be requested to answer a set of items 
designed particularly for them and vice versa. 

9. Would you like to see a course on multilith 
(offset) printing returned to the school's 
curriculum? 

( ) Yes 
( ) No 

( ) No opinion 

The word "returned" indicates the course had been 

offered previously, but dropped for some reason. If two or 

three years have passed, it cannot be assumed that teachers, 

students, and parents will remember the course. A brief 

description of the course should be given and the word 

"returned" should then be clarified by stating when and 

perhaps why the course was dropped. 

10. Have you ever had the opportunity to use your 
metal shop training after graduation? 

( ) Yes 
( ) No 

The background of each respondent is important for 
obtaining valid information from this question. When polling 
the sample, persons with metal shop experience must be separ- 
ated from those with none. In this manner, the response can 

be better interpreted by subgroup analysis. However, the 
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above question is a very poor one. What is meant by "ever 

used?" There are simply too many interpretations to the 

question* If the question is needed, reword an entire series 

of questions which identify clusters of skills associated 

with those taught in metal shop. 

Does the question need to be more concrete, specific , 

and closely related to the respondent's personal experience ? 

Questions must be structured to elicit characteristic 

responses. The entire questionnaire must be concise enough 

so that the respondent will not be required to use time 

"guessing" about question intent. To insure accuracy and 

ease in responding, questions must be written in a sequential 

manner. For example, if the question below were the only 

one asked the results could be very erroneous since it would 

require a series of statements relating to goals and objectives 

of student personnel services. 

11. Do you think the career counselor position is 
important to the school? 

{ ) Yes 
( ) No 

( ) No opinion 

For preciseness, in addition to asking students if 
they value the career counselor, questions could be added 
to determine the number of times students visited career 
counselors during a one-year period. A series of these 
types of questions allow assessment of respondents* behavior 
in addition to attitude. 
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12* List your last Saturday's activities. Start from 
the time of awaking, proceed through the day, and 
conclude with bed time. Be as precise as possible. 



Items such as this are often used for assessing values and 
behavior. A student may report that reading is enjoyed, but 
if time is not spent actually reading, it probably is not 
valued. It may be noted that questions such as Number 12 
above are currently under scrutiny for invasion of privacy. 
It is used here as an example of a method that ascertains 
what the students are actually doing. For even greater 
preciseness, students should be asked if the activities listed 
are typical of most of their Saturdays. The latter might 
be considered a dual set or couplet question methodology. 

Is the question free from spurious specificity ? 
Usually, questions that lack specificity cause problems. 
However, lengthy questionnaires with too many specific items 
may cause loss of interest, create confusion, or prevent the 
completion of the instrument. Do not overwhelm respondents 
with the content of the questionnaire. Questionnaires are 
almost always completed on a volunteer basis. 

A list of 200 occupations which requires that the 
respondents mark the ones of interest to them is a confusing 
and boring task. A better method would be to group the 
occupations into a series of manageable categories and ask 
for preferences within job groupings. 
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When respondents are asked to rank a list of items, 
the list should not contain more than 10 items. Lists 
over 10 become rather difficult to judge. Most people are 
not generally willing to complete one set of extensive 
rankings. It is better to compile two or more lists. The 
respondents would be asked to rank order each list. In 
general, we consider five or six items as the maximum number 
to rank order. 

Question Wording 

Can the question be misunderstood? Be aware of 
educational jargon when conducting surveys. Teinns such 
as closed campus, module, team teaching, accountability 
may mean nothing to some and too many things to others. 
Educators are frequently insensitive to the jargon and 
concepts which they use. Most educational terms are totally 
unknown or misunderstood by school patrons. Language and 
vocabulary limitations of the clientele are important 
considerations that must be realized when developing a 
survey instrument. Words that have multiple or vague 
definitions can invalidate a questionnaire. 

13. What do you enjoy most about teaching? 



This open ended question may elicit a majority of "the summer 

vacation" responses. The responses would be honest, but 

hardly appropriate to the intent of the question. If there 

are doubts about the wording of a question, write it in 
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different ways and field test the questions on a small group 
to determine which version elicits the data in the desired 
form. Better yet, provide a series of forced response or 
rank order items. 

Is the question likely to be objectionable? Topics 
such as age, income, religion, personal habits, and personal 
likes and dislikes are usually classified as "sensitive." 
Offensive t^rms such as "school dropout" and "poor grades" 
should not be used. Respondents tend to avoid questions 
that show them in an adverse situation. Topics such as 
these should be omitted if possible, but if the information 
is necessary wording should be changed to make the topic 
less offensive. Examine the next two examples. 

14. How old are you? 

Students may not object to this question, but the 
general public may. This sensitive information can be 
obtained by using a response continuum with five-year or 
ten-year intervals. It is less offensive for the respondent 
to put a check mark in the 35-39 category than it is to write 
39 years of age. 

15. Do you have a discipline problem in vocational 
classes? 

{ ) Yes 

( ) No 

( ) Neutral 

The results from this question will invariably be 

less than accurate because the respondent may not be aware 

or may not want to admit that a discipline problem exists. 
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This information can be obtained more accurately by asking 
questions such as, "Over the last five school days, have you 
had a child disrupt your class to the point where punitive 
action was necessary: e.g., sent to office or being sent out 
of the room?" "If yes, how many children?" A teacher that 
sends four or five students out of the room in a week's time 
may be defined by the researcher as having a discipline 
problem. By asking situational questions, the term "discipline 
problem" is more operationally defined. Comparisons can then 
be made with other respondents. 

Wording the question to produce results . There have 
been a niaitiber of studies on personal and impersonal wording 
of questionnaires. These show that results do differ with 
just a slight personal change in the wording. Versions of 
personal and impersonal questions follow. 

16. To what extent are career resources in the library 
satisfactory? 

) Very satisfactory 

) Satisfactory 

) No opinion 

) Unsatisfactory 

) Very unsatisfactory 

17. To v;hat extent do you think career resources in the 
library are satisfactory? 

) Very satisfactory 

) Satisfactory 

) No opinion 

) Unsatisfactory 

) Very unsatisfactory 

When answering question number 16, the respondent may 

consider other teachers' and persons* opinions along with his 
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or her own. Question 17 specifically asks for a personal 
response. Use personal and impersonal wording to elicit the 
response most appropriate to the purpose of the study. 

There are no definitive rules for deciding whether 
to use a more direct or more indirect approach. The specific 
situation must be interpreted by the researcher. Consider 
each question individually in making decisions, keeping in 
mind such things as sensitive topics, personal attitudes 
versus public attitudes, and respondent background. The 
following sets will exemplify the approaches. 

18. When was the last time a woman in your organization 
was promoted? , 

19. Do you believe there is sexual discrimination in 
promotions in your organization? 

{ ) Yes 
{ ) No 

{ ) No opinion 

The first, indirect, approach will provide an objec- 
tive answer by which the researcher can systematically and 
objectively define the existence of sexual discrimination. 
The second, direct, question ascertains subjective perceptions 
and would be more appropriate for determining opinions about 
discrimination in promotional practices. Further, opinions 
may be in error, especially if there is a very high rate of 
personnel turnover . 

The series of questions which follow illustrate the 
difference between using a direct or indirect approach to 
seeking information. 
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20* Why were you fired? 

21. How would you rate your relationships with your 
managers? 

( ) Very good 

( ) Good 

( ) Average 

( ) Poor 

( ) Very poor 

22. What percentage of your fellow employees did not 
get along well with you? 

( ) 100% 

( ) More than 50% 
( ) 50% 

( ) Less than 50% 
( ) None 

23. How was your work rated by your manager? 

( ) Very good 

( ) Good 

( ) Average 

( ) Poor 

( ) Very poor 

Question 20 and the three subsequent questions were 
designed for the purpose of ascertaining the reasons for 
which people are dismissed from their jobs. The responses 
obtained by using a direct approach in question 20 will 
probably not be returned or will be vaguely or perhaps 
dishonestly answered. By asking a number of indirect 
questions (21, 22, and 23), accurate and more precise infor- 
mation can be obtained. ^ 

A well constructed set of indirect questions will 
provide data from which a set of logical inferences may be 
drawn. 
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Question Placement 

Logical leads > Where a question is placed in the 

questionnaire is very important since the respondent must 

recall information, make opinions or perhaps respond to some 

items which may be personal, e.g., age, schooling, income. 

The questionnaire should reflect a well organized outline with 

each set of items addressing a distinct topic. In this manner 

a respondent will tend to respond in an easy fashion. 

When seeking chronological information, arrange the 

items so that the respondent begins with the present. For 

example, if one were seeking an employment profile the' question 

might ask: "List the jobs which you have held chronologically, 

beginning with your present one." If such an item were to be 

a chief source of information, then the investigator might 

prepare a form which listed the job, year begun, year quit, 

and other relevant descriptions. 

When sensitive areas are assessed, it is important 

that a conducive psychological order be established- If 

school "dropouts" are initially asked, "Why did you quit 

high school?" the few responses that might be returned will 

probably be vague or inaccurate. In contrast, by asking 

questions that lead the respondent through non-threatening 

items which lend themselves to easy responses, the respondent 

becomes less threatened by the more sensitive questions. 

Following the latter design will increase truthfulness in 

responding and the number of questionnaires returned. 
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Question placement s Place the easy, non-threatening 
questions at the beginning of the questionnaire. These 
would include sex, job, age, marital status, number of 
children, whether the spouse works or not, and other non- 
threatening demographic data. Note that these items are not 
very sensitive, although they are personal. As a category, 
these are called the "informational" questions. The 
informational questions are short, unoffensive, and allow the 
respondent to become accustomed to completing items. Once a 
rapport has been established, the respondent is more committed 
to the tasks and more sensitive questions are tolerated. 

If a researcher desires to build some reliability 
checking items, they may be included in each section or 
topic. This usually requires that a major opinion-seeking 
item be reworded in two or three different forms. In this 
manner the researcher can determine if the respondent is 
consistent when responding. (If there is little consistency, 
that questionnaire may have dubious worth.) 

Response Modes 

Forced or Open Response Categories 

Forced response questions include response categories 
which are predetermined by the researcher. The categories 
of forced response items which require the respondent to mark 
one category only must be exhaustive (include all possible 
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responses) and mutually exclusive (each response excludes 
all other possible responses, i*e,/ if a respondent is male — 
he cannot be female). In contrast, the categories of forced 
response items which require the respondent to mark al3 
appropriate (more than one) categories must be exhaustive but 
not mutually exclusive. These items should provide alterna- 
tives for multiple responses, but yet, differentiated from 
each other. When it is not feasible to list all possible 
responses, an "other-please specify" category can be used. 
Use of this category, however, should be limited due to 
difficulty in tabulating and interpreting results. Examples 
of these two types of forced response items follow. 

Exhaustive and mutual exclusive categories . One 
illustration of a mutually exclusive category is listed 
below: 

24. Every high school student should be taught a 
marketable skill. 

Strongly Agree 

Agree 

Undecided 

Disagree 

Strongly Disagree 

The respondent can logically mark only one category, 
because the categories are mutually exclusive. Note also 
that an equal number of positive and negative responses are 
presented. 

Categories which allow more than one response . In 
some cases the investigator may want a check-list construction 
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to determine broad categories for perhaps, future follow-up. 

One example follows. 

25. High school students should be taught which of 

the following marketable skills? Mark any number 
of choices. 

( ) Bookkeeping 

( ) Carpentry 

( ) Plumbing 

( ) Cashier Operator 

( ) Truck Driving 

( ) Computer Programming 

( ) Other-please specify 

It would be impossible to list all of the skills 
that people may think should be taught in high schools, 
therefore the "Other" category is provided. Respondents have 
a choice to mark any number of responses. The teaching of 
one marketable skill does not exclude the teaching of other 
marketable skills . 

Open ended questions . An open ended question does 
not include response categories. The respondent is free to 
select the response form. There are at least three reasons 
for using an open ended question: (1) to probe an idea 
further, (2) to accommodate categories which are incomplete 
or inadequate in a forced response list, and (3) to provide 
projective types of situations. Open ended questions are 
inappropriate when forced response categories can be 
formulated. Questions that can be answered yes or no should 
always be in a forced response format. Tabulating results 
and reaching meaningful generalizations are much easier with 
forced response questions. 
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Open ended questions require a more complex coding 
system than forced response questions* Coding is explained 
in detail under its own heading in Chapter 4. Be aware ^ 
however, at this point that tabulating results requires 
examination of each respondents' answers individually. 
Categories that are characteristic of the responses must be 
developed into which each of the responses must be classified. 
In contrast the categories of forced response questions are 
already developed for each respondent. Obviously, analyzing 
open ended questions is more time consuming than analyzing 
forced response questions. In some cases, the investigator 
may combine both types of questions. An example of a mixed 
open ended question follows. 

26. Do you feel every high school student should be 
taught at least one marketable skill? 

( ) Yes, please elaborate 

( ) No, please elaborate 

Another version of the same question as an open ended 

question could be framed. 

27. Recently it has been written that every high school 
student should be taught at least one marketable 
skill. V?hat are your reactions to that statement? 



Is the form of response easy, definite, uniform, and 
adequate for the purpose ? Be certain that the respondent is 
able to reply in a manner that reflects precisely what is 
wanted. The entire questionnaire must be structured so that 
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there will be little chance for confusion* Not only will 
respondents cooperate with instructions but a well structured 
questionnaire lends itself to easy tabulation. Compare the 
first set of questions with the second set in the following 
examples. 

Circle your sex: Female Male Check your 

ethnic background, Black or Afro-American 

Oriental or Asian American Indian or other native 

American Spanish Surname Caucasian (Other 

than Spanish surname) Other (please specify) 

Please write your year of birth 

1. Sex: (1) Female (2) Male 

2. Ethnic Background: 

(1) Black or Afro-American 

(2) Oriental or Asian 

(3) American Indian or other Native American 

(4) Spanish Surname 

(5) Caucasian (other than Spanish surname) 

(6) Other (please specify) 

3. Your Age 

(1) Under 20 

(2) 20-24 

(3) 25-29 

(4) 30-34 

(5) 35-39 

(6) 40 or over 
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The first set of examples is too difficult to follow 
and causes confusion* Further, the appearance gives the 
impression that not much effort or organization has gone into 
the instrument's preparation (the term instrument is used 
interchangeably with questionnaire). The second set of 
examples displays each question and response in a neat, 
organized fashion. Tabulation of the responses in the second 
example would also be accomplished much easier than for the 
first. 

In Appendix A there are displayed a series of 
question formats. Appendix A also illustrates how questions 
may be eirranged for a wide variety of instrument styles. 

' We have attempted to stress throughout this chapter 
that good questionnaires are well planned and that each 
question or response item be very carefully examined. The 
preplanning needed to construct a survey instrument pays off 
with valid data and high response rate. In the next chapter 
we address the topic of preparing forced response questions. 
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CHAPTER 3 



FORCED RESPONSE QUESTIONS 

Establishing Appropriate Scales 
Forced response questions typically represent one of 
three measurement scales (systems of numerical notation) . 
It is important to distinguish among these scales when coding 
responses or conducting statistical analysis. A working 
knowledge of these scales will also aid in designing appro- 
priate questions. 

Nominal Scales 

One type of forced response question represents the 
nominal or "naming" scale > The response categories of a 
nominal item are basically non-numerical in their relationship. 
This scale identifies rather than measures. Questions 
representing a nominal scale are usually designed to gather 
factual (objective) information about the respondent. For 
example: 

1. I am 

(1) Male 

(2) Female 

Note that the response categories must be both 

exhaustive and mutually exclusive when the respondent is 
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required to mark only one category. The categories of the 

preceding example are exhaustive because the respondent must 

be either male or female, and the categories are mutually 

exclusive because the respondent is not a member of both. 

In addition to factual information about the respondent, 

one may gather factual information, for example., about existing 

programs or school policies. All questions which require the 

respondent to answer "yes," "no," or "maybe" represent a 

nominal scale. For example: 

2. Are vocational agriculture courses 
now being taught in your school? 

(1) Yes 

(2) No 

(3) Do not know 

Ordinal Scales 

The ordinal scale , which represents another type of 

forced response question, is usually used to gather both 

factual information and respondent opinion. The ordinal 

scale indicates a rank order relationship among the response 

categories of a question; however, it does not reveal how 

much difference there is between the categories. For example: 

3. Which of the following best compares 
your teaching load this year to your 
assigned teaching load for next year? 

(1) I will teach fewer classes 

next year. 

(2) I will teach the same number 

of classes next year. 
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(3) I will teach more classes 

next year. 

The response categories of this question have a 
definite mathematical relationship; "fewer" is less than 
"same" and both of these categories are less than "more." 
However, the researcher cannot assume that there are equal 
differences between categories "few" and "same," and 
categories "same" and "more." One respondent may be 
assigned to two fewer classes for next year, whereas another 
respondent may be assigned to one more class for next year. 
Because the categories do not request the information how 
many "fewer" or how many "more" classes; equal distances 
cannot be assumed. 

Another method of gathering opinions on a systematic 
basis requires that respondents rank the categories of a 
question according to their preference. Rank order questions 
also represent an ordinal scale. For example, a respondent's 
first preference may reflect weak opinions that are rather 
difficult to differentiate. Consequently, there can be a 
greater difference in preference between the first and second 
ranked items, than there is between the second and third. 
Because the researcher cannot determine the amount of 
difference between categories, unequal differences are assumed. 
The following example illustrates a rank order question which 
seeks to obtain factual information: 
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4. If no vocational agriculture courses are planned 
for the immediate future, please rank the first 
and second reasons which best describe why the 
courses will not be added* Place a "1" in front 
of the primary reason. Place a "2" in front of 
the second most important reason. Please do not 
rank any other categories. 

(1) Not enough student interest 

(2) No certified Vo-Ag teacher 

(3) No need for agriculture training in 

our school 

(4) Lack of facilities 

(5) Lack of money 

(6) Levy failures 

(7) Other subjects are mor^e important 

(8) Other, please list ^ ^^^^ 



The categories of a ranked item do not inherently 
possess a mathematical relationship. However, when the 
respondent ran].^ the categories a mathematical relationship 
is established. 

Likert scales . The most widely used ordinal scale 

among survey researchers is called the Likert: scale , after the 

founder, Rensis Likert. Questions which require rating 

usually represent Likert scales. Such scales, used primarily 

for assessing opinions, are usually composed of five or more 

response categories: e.g.. Strongly Agree, Agree, Undecided, 

Disagree, Strongly Disagree; or some such continuum. The 

following examples represent Likert-type scale items. 

5. More vocational courses are needed in high school. 

( ) Strongly Agree 

( ) Agree 

( ) Undecided 

( ) Disagree 

( ) Strongly Disagree 
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6. I would encourage the school board to promote 
programs aimed at providing job skills to the 
physically and mentally handicapped. 

( ) Encourage very much 

( ) Encourage 

( ) No opinion 

( ) Discourage 

( ) Discourage very much 

7. Rate yourself as a consumer education teacher: 

( ) Excellent 

( ) Good 

( ) Average 

( ) Poor 

( ) Unsatisfactory 

8. How well do you like attending workshops as 
compared to other instructional procedures? 

( ) I like workshops much better 

( ) I like workshops a little better 

( ) I like all instructional procedures about 

equally well 
( ) I like other instructional procedures a 

little better 
( ) I like other instructional procedures much 

better 

9. Students call vocational education teachers by 
their first names. 

( ) Almost always occurr 
( ) Frequently occurs 

( ) Approximately equal in occurrence and 

nonoccurrence 
( ) Infrequently occurs 
( ) Almost never occurs 

10. Would you tend to favor a new income tax as a 
way of reducing school taxes? 

( ) Strongly favor 

( ) Tend to favor 

( ) Not sure 

( ) Tend to oppose 

( ) Strongly oppose 
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11. Do you feel that as a vocational counselor you could 
now effectively use the tests which were covered 
in the workshop? 

) Definitely yes 

) Probably yes 

) Uncertain 

) Probably not 

) Definitely not 

12* Extracurricular activities, such as DECA, VICA, 
and FFA, in high school were: 

) Very important 
) Important 
) Undecided 
) Unimportant 
) Useless 

The respondent is requested to select only one category of a 
Likert scale item. The categories must be exhaustive and 
mutually exclusive. The above models and all others in this 
manual are for your use. Please feel free to adapt them as 
scales as prepared for various surveys. 

Interval Scales 

The most sophisticated scale used by survey researchers 

is the interval scale . The term "interval" connotes a rank 

order relationship and equal differences between categories. 

The latter characteristic distinguishes the interval from the 

ordinal scale. The following question uses the interval scale. 

13. How many secondary school students attend your 

high school (grades 9-12 or 10-12) for 1974-1975? 

(1) 1,000 or more 

(2) 750-999 

(3) 500-749 

(4) 250-499 

(5) 249 or less 
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The above categories have a 250 point interval, making 
the categories equi-distant . Interval items are frequently 
used to obtain quantitative information. 



Transforming Items 

Nominal, ordinal, and interval daca may require 
different coding techniques and forms of statistical analysis • 
Therefore, ib is important to know precisely which scale a 
forced response question represents. It is an acceptable 
procedure to regroup data by converting from a higher to a lower 
order of measurement (from interval to ranked to nominal). 
But, conversion from a lower to a higher order is not ordinar- 
ily done. For example, consider age data and how they can be 
described using nominal, ordinal, and interval scales. 

Interval scale . 14 . Your age group is : 

(1) 15-19 

(2) 20-24 

(3) 25-29 

(4) 30-34 

(5) 35-39 

(6) 40-44 

(7) 45-49 

(8) 50-54 

(9) 55-59 

(10) 60 or over 



Ordinal scale . 15. Your age group is: 

(1) Adolescent 

(2) Adult 

(3) Middle age 

(4) Retirement age 



Nominal scale. 16. Were you born during the baby 

boom, i.e. , between 1946 and 1950? 

(1) Yes 

(2) No 
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In these illustrations, the interval scale includes 
the information sought in both the ordinal and nominal forms. 
By using the interval form, more precise information can be 
obtained, plus the option of regrouping the data into more 
general categories. To change the item from interval to 
ordinal data, and assign the "under 19" category to the 
adolescent group, categories, "20-24," "25-29," "30-34," 
"35-39" to the adult group, categories "40-44," "45-49," 
"50-54" to the middle age group, and the "60 and over" to 
the retirement age group. Similarly, to change the item from 
an interval to a nominal item; assign category "25-29" to the 
yes group, and all other categories to the no group. The 
reverse procedure, changing nominal to interval data, cannot 
be accomplished because the nominal response categories are 
less specific and thus less informative than the interval 
categories. 

Self-Practice Items 
The models below are of different scales and have 
been formulated for your practice. Determine the scale which 
each item represents. The appropriate responses follow after 
item H. 

A. I plan on going to a community college after I 
graduate from high school. 

{ ) Yes 
{ ) No 
( ) Maybe 
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Vocational Education is valuable for girls. 

( ) Strongly agree 

( ) Agree 

( ) Undecided 

( ) Disagree 

( ) Strongly disagree 

Dl^ you have any males under 18 years of age residing 
at your home now? 

( ) Yes 
( ) No 

My ability to explain "how" to do things is: 

( ) Very good 

( ) Good 

( ) Average 

( ) Poor 

( ) Very poor 

How many vocational teachers taught at your high 
school during 1974-75? 

( ) 1 
( ) 2 
( ) 3 
( ) 4 

( ) 5 or more 

Which of the following best compares this year's (1974- 
75) vocational agriculture curriculum to next year's 
(1975-76) planned vocational agriculture curriculum? 

( ) More classes will be offered next year 
( ) Same for both years 

( ) Fewer classes will be offered next year 

Next year's vocational agricultural curriculum will 
cause which of the following to happen to your 
staffing pattern? 

( ) No change 

( ) More certified Vocational Agriculture teachers 

will be needed 
( ) Certified Vocational Agriculture teachers v;ill 

teach other subjects 
( ) Certified Vocational Agriculture teachers will 

be transferred to another school 
( ) Other 
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H. Why are you attending State University? Please rank 
the first and second choices which best describe why 
you are attending. Place a "1" in front of the most 
important reason. Place a "2" in front of the 
second most important reason. Please do not rank 
other categories. 

( ) Parents suggested attendance 
( ) Relative attending SU suggested attendance 
( ) High school counselor or teacher suggested 
attendance 

( ) Friend attending SU suggested attendance 
( ) Cost of attending SU was most reasonable 
( ) SU was conveniently located for attendance 
( ) I considered SU an outstanding university 
( ) I would prefer attending another university, 

but I had no other choice 
{ ) University Day sold me on SU 



KEY A. Nominal 

B. Ordinal 

C. Nominal 

D. Ordinal 

E. Interval 

F. Ordinal 

G. Nominal 

H. Ordinal 
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CHAPTER 4 



CODING SURVEY ITEMS 

Forced Response Codes 
Data which are obtained from interviews and question- 
naires are not necessarily usable in the exact form in which 
they are collected. Data must usually be converted to some 
quantitative form for analysis and display. The converting 
process is called scoring or coding. Most forced response 
questions can be preceded prior to questionnaire distribution 
with ease. The researcher simply assigns, in advance, a 
score to each response category of the questions. Then each 
response can be immediately and directly converted into a 
score (quantified) in an objective, consistent, and systematic 
manner. 

Coding nominal data . The response categories of a 

nominal question do not have any quantitative relationship to 

one another. Therefore, numerical symbols for coding are 

assigned arbitrarily. Consider the following example: 

1. I plan on getting a full time job in my community 
after high school graduation. 

(1) Yes 

(2) No 

(3) Undecided 
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We have assigned a code of (1) to the response "yes, 
a code of (2) to t h e response "no/' and a code of (3) to 
the response "undecided." This coding system does not imply 
that the response "yes" (code 1) is less than the response 
"no" (code 2) . The numerical codes for nominal data simply 
identify mutually exclusive categories. Nominal data items 
can then be analyzed by a simple count of the number of 
respondents who marked each category, e.g., the number of 
respondents who marked category 1, the number who marked 
category 2, and the number who marked category 3. The above 
coding system would be used for all questions of similar 
construction in the questionnaire. In that manner a uniform 
coding system is maintained. 

Coding ordinal and interval items . The response 
categories of ordinal and interval items do have a quantita- 
tive relationship, therefore, numerical symbols should be 
assigned consistently. Consider the following example: 

2. Which of the following best compares this year's 
(1974-75) vocational curriculum to next year's 
(1975-76) vocational curriculum? 

(1) Fewer classes will be offered next year 

(2) Same for both years 

(3) More classes will be offered next year 

For this example, "fewer classes" (code 1) is less than "same 
(code 2) , and code 1 and code 2 are both less than "more 
classes" (code 3) . Ordinal data of these types and interval 
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data can be analyzed by counting the number of respondents 

who marked each category. Note in the example that the date 

is included in the question to aid in precise and explicit 

interpretation . 

Ranked items are scored individually by each 

respondent who is requested to number a set of categories 

according to some criterion. Consider the following example: 

3. Did you attend University Day because: (Please 
rank the following five items according to your 
reasons for attending. Place a "1" in front of 
the primary reason, a "2" in front of the second 
most important reason, and so forth until you have 
ranked all five items. You may use a number only 
once. 



1. Parents suggested attendance 

2. High school counselor suggested attendance 

3. Saw advertisement on local TV which 

influenced you 

4. Friends suggested attendance 

5. Saw advertisement in high school 



Each ranked category requires one response, and can be viewed 
as a separate question. In the above example, each respondent 
scores every category. Ranked items can be analyzed by 
determining the mean rank for each response category. 

Coding Likert scale items . The Likert scale is most 
frequently used to obtain ordinal data. The instrument is 
designed so that a respondent selects one of five categories 
that best describes an opinion toward the question. The 
response categories of a Liker b scale item do have a quantitative 
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relationship and consecutive scores must be assigned to 

consecutive categories* For example: 

4, My children are receiving adequate vocational 
counseling in high school. 

{ ) Strongly ( ) Agree ( ) No ( ) Disagree ( ) Strongly 

Agree Opinion Disagree 

If you assign a code of "one" to Strongly Agree, then you must 

assign a code of "two" to Agree, a code of "three" to No 

Opinion, a code of "four" to Disagree, and a code of "five" 

to Strongly Disagree. If you assign a code of "five" to 

Strongly Agree, then you must assign a code of "four" to 

Agree, a code of "three" to No Opinion, a code of "two" to 

Disagree, and a code of "one" to Strongly Disagree, The 

order must be followed consistently throughout your instrument. 

Coding subscales of Likert scale items . Often a 

questionnaire will contain a number of subscales, each of 

which measures a different aspect of the total scale* For 

example, if the purpose of the survey is to measure the community 

need for a specific aspect of a vocational education program, 

part of the questionnaire may ask four or five questions pertaining 

to the respondent's attitude toward vocational education in 

the comprehensive high school. These four or five questions 

comprise a subscale, and are viewed as a group because each 

question attempts to discover the same attitude from different 

perspectives. When you rely on more than one question to 

assess an attitude, the response reliability is greater. 
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Questions composing a subscale should be distributed through-- 
out the questionnaire so that a respondent must consider each 
question individually rather than simply falling into a 
"response pattern syndrome" or "mind set." 

A few of the questions composing a subscale should 
be worded such that if the respondent agrees with a statement, 
e.g., the present vocational courses, there will be a positive 
response. Other questions in the same subscale, however, 
should be so worded such that the respondent who is pleased 
with the present vocational courses will respond negatively. 
Reverse wording alleviates biasing a subscale and provides a 
means by which to ascertain respondent consistency. For 
example, the follov;ing pair of questions are reversely 
worded . 

5. Students enrolled in a vocational course in the 
comprehensive high school are well prepared to 
seek a job in the area. 

( ) Strongly ( ) Agree ( ) Undecided ( ) Disagree ( ) Strongly 



6. Students who receive their vocational training in the 
comprehensive high school tend to be inadequately 
trained . 

( ) Strongly ( ) Agree ( ) Undecided ( ) Disagree ( ) Strongly 



A respondent who strongly favors the vocational program will 
mark "strongly agree" to the first question and "strongly 
disagree" to the second question. These questions are called 
reversed items. 



Agree 



Disagree 



Agree 



Disagree 
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The purpose of using and analyzing a subscale is to 

assess the overall opinion toward an issue. Affirmative 

< 

responses toward the issue must be coded in one direction 
while all negative responses toward the issue must be 
coded in the opposite direction . In Table 1 is an illus- 
tration of the coding procedure for reverse items. 



Table 1. Scoijing of Reversed Questions 







Scale^-^Scores 




Affirmative 
Response 
(Question 5) 


1 


2 3 4 


5 




strongly 
Agree 


Agree Undecided Disagree 


Strongly 
Disagree 


Negative 
Response 
(Question 6) 


5 


4 3 2 


1 



In this example, code 1 in questions 5 and 6 refers to a 
positive response toward the issue, while code 5 in questions 5 
and 6 refers to a negative response to the issue. The respondent 
who highly favors the vocational education program will receive 
a score of two, while the respondent who highly disfavors 
the program will receive a score of 10. If the coding were 
not reversed for these two questions, the respondent who 
highly favors the program would receive the same score as the 
respondent who highly disfavors the program. To evaluate the 
group of questions as a subscale, you must code according to 
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the response toward the issue that the subscale as a whole 
is attempting to assess. Practice coding the following 
items: The responses are listed on the following page. 

7. I enjoy myself most of the time in school. 

( ) Strongly ( ) Agree ( ) Undecided ( ) Disagree ( } Strongly 
Agree Disagree 

8. When I am in school I usually feel unhappy. 

( ) Strongly ( ) Agree ( ) Undecided ( ) Disagree ( ) Strongly 
Agree Disagree 
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Correct Responses— Two Alternatives 

Scale Scores 



(Question 


7) 


1 


2 3 


4 


5 






Strongly 
Agree 


Agree Undecided 


Disagree 


Strongly 
Disagree 


fOup «?t Ton 










1 


(Question 


7) 


5 


Scale Scores 
4 3 


2 


1 






Strongly 
Agree 


Agree Undecided 


Disagree 


Strongly 
Disagree 


(Question 


8) 


1 


2 3 


4 


5 



open Response Codes 

Coding open ended items > Open ended items must also 
be quantified to analyze and summarize questionnaire or 
interview data. Coding an open ended question involves the 
superimposing of a structured format to a free or unstructured 
response. For efficiency, researchers often establish 
preceded response categories to account for responses of 
open ended questions. The extent to which preceding is 
possible is an indication of the extent to which the question 
is likely to yield relevant information. To avoid coding 
unreliably, open ended responses should be ceded by at least 
two judges. It is the responsibility of the judges to 
categorize the unstructured responses according to the 
predetermined coding system. The coders must be trained in 
advance by the researcher. 

The following question is an open ended item: 

9. At what grade levels do you think children should 
be taught about careers and the world of work as 
a part of the school program? 

The respondent is free to express an opinion in any manner. 

One respondent may write a paragraph describing the background 

information that led to the decision, another may write one 

summary word describing the opinion. 

To establish preceded categories for an item, a 

researcher must knew why the question is being asked and 

anticipate types of responses. The following are at least 

two possible ceding systems for the above item: 
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Grades 1-3 Elementary school 

Grades 4-6 Junior high school 

Grades 7-8 (or) Senior high school 

Grades 9-10 
Grades 11-12 



Coding checklist items . Checklist items are a 

reasonable alternative to open ended items because they give 

the respondent more latitude, and also provide preidentif ied 

categories which are usable for coding. A checklist question 

is a special form of a forced response item which allows for 

multiple responding. For example: 

10. Children should be taught about careers and the 
world of work as a part of the school program at 
which grade level (s). (Check as many as you 
desire. ) 

10.1 Grades 1-3 

10.2 Grades 4-6 

10.3 Grades 7-8 

10.4 Grades 9-10 

10.5 Grades 10-12 



To code multiple response items, each category must be viewed 
as a separate question to which the respondent may answer 
"yes" or **no." The respondent has the option to mark as many 
or as few categories as desired. The categories are not 
mutually exclusive and therefore the respondent considers 
each category in isolation. Within each category, a negative 
response may be coded "1" and a positive response may be 
coded "2." Whatever the coding system that is used must 
then be maintained for the entire study to assure uniform 
reporting. 

Occasionally, checklist items will include a category 

titled "Other, please specify." If the researcher is 
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interested in analyzing these responses^ the category must 
be approached like an open ended item. Responses must be 
assigned to subcategories and each subcategory must be 
analyzed individually • 

Coding is a necessary procedure for analyzing and 
displaying survey data. A working knowledge of the types of 
forced response questions aids in developing a consistent 
and accurate coding system. Preplanning will not only make 
the coding procedure more efficient, but will improve the 
accuracy of the instrument. Included in that preplanning 
is care in constraction of items to avoid any possibility 
of misinterpretation or bias. Chapter 5 will present a 
discussion of those concepts. 
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CHAPTER 5 



PREVENTING BIASED RESULTS 

Wording the Questions 
Researchers often hold preconceived conclusions or 
assumptions pertaining to the research topic, respondents, 
or hypothesized results. These assumptions can cause a 
"bias" to be automatically built into a survey instrument* 
Built-in biases and inappropriate sampling almost always 
yield biased results. Researchers with a vested interest 
must be cautioned not to "load" a questionnaire by favoring 
one side of an issue or by devising questions which force 
respondent bias. Loading a questionnaire in any manner does 
not allow respondents the opportunity to disclose their true 
feelings about an issue. Any deliberate loading or biasing 
is totally unethical and unprofessional . It is also 
professionally dangerous: you'll probably be caught and 
discredited! 

Many career or vocational courses include units which 
teach skills on how to find a job. An instructor in such a 
course may wish to assess job hunting techniques used by 
unemployed high school graduates. A questionnaire might be 
used, with one item on the questionnaire asking: 
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1. Which of the following sources do you use for 
finding job openings? (You may check more than 
one source.) 

] 1.1 State Employment referrals 

] 1. 2 Friends and relatives 

] 1.3 Newspaper ads 

] 1.4 Various personnel office employees 

] 1.5 Private employment agencies 

] 1.6 Schools 

] 1.7 Other (please specify) 



The built-in bias or probable assumption is that all 
unemployed p^^rsons are actively seeking work. If a number 
of respondents check only "Friends and relatives," the item 
may imply the distorted conclusion that respondents are 
unaware of useful sources of employment leads. However, 
some unemployed do not actively seek work and, therefore, 
sources of employment leads may be known but are not being 
utilized. 

In this case, to assess the knowledge about job 

hunting rather than actual behavior, the item can be rewritten 

as is illustrated below. 

2. If you wanted a job, which of the following sources 
would you use to find work? (Check any number that 
you would use. ) 

[ ] 2.1 State Employment referrals 
[ ] 2.2 Friends and relatives 
[ ] 2.3 Newspaper ads 

[ ] 2.4 Various personnel office employees 
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[ ] 2.5 Private employment agencies 
[ ] 2.6 Schools 

[ ] 2.7 Other (please specify) 

The item above places all respondents into a neutral frame 
of reference which seeks respondent knowledge. Unwarranted 
assumptions about the respondents are in this manner avoided. 
The important lesson to be learned here is that the simple 
framing of the question may reflect assumptions about 
respondents. 

One ought to be aware of another rather subtle bias 
that can be built into an instrument; the "yes tendency," or 
the "response pattern syndrome." While responding to a list 
of questions, respondents may inadvertently have "mind set" 
by continuously answering to the positive. This tendency can 
be partially offset by including inverse or reverse questions. 
This question constructing technique requires the use of 
pairs scattered throughout the questionnaire. The technique 
helps to avoid the "yes tendency" and also acts as a means of 
checking respondent consistency on some issue or point. 

Questions that are direct opposites to others tend to 
be redundant, but related ideas can be used to construct 
reverse questions. The following pair of questions might be 
used in determining attitudes toward a longer school year. 
3. Do you favor a longer school year? 
[ ] 3.1 Yes 
[ ] 3.2 No 
[ ] 3.3 Undecided 
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4. If the school board approved a shorter school 
year, would you endorse such a decision? 

[ ] 4.1 Yes 

[ ] 4.2 No 

[ ] 4.3 Undecided 
In the above set, if "yes" is marked for both questions, one 
can conclude that the respondent has fallen into a "yes 
tendency" or that the respondent is inconsistent in opinion. 
A field test of a questionnaire may identify items that cause 
a "yes tendency." The researcher may then make adjustments for 
its reduction. Obviously the questions would be separated 
and would not follow each other. 

Sampling Considerations 
Surveys that are conducted by vocational educators 
often involve a known group of people (finite population) . 
For example, all the public high school principals in a 
county or all seniors of one school might be the target 
population. In such cases the persons to be surveyed (sample 
of respondents) might include the entire target population. 
If the sample does not include the entire target population, 
then conclusions may be drawn related to the sample only. 
Unless appropriate statistical tests are used, predictions 
cannot be made with great certainty about the entire target 
population. For example, if a survey of 100 seniors is 
conducted, the investigator cannot automatically predict 
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that all seniors in a school district follow the patterns 
of those sampled • 

Surveying the entire population of a comparatively 
small group eliminates the chance of sampling bias and is the 
preferred procedure. Improper sampling techniques almost 
always lead to biased surveys. However, sampling techniques 
are not difficult to use if precautions are taken. Volunteer 
samples often result in a biased study. When samples are 
drawn from any population, all members of the population must 
have an equal chance of being selected. Simple random 
sampling or stratified random sampling are two basic 
recommended techniques. The characteristics that may cause 
a bias in these sampling techniques will be discussed. 

Random Technique 

Simple random sampling is effective if a listing of 
the total population has been prepared. A telephone book is 
not a listing of the total population because such a list 
usually discriminates against the pcor and is inaccurate 
since a large number of persons change residences, leave town, 
or have new numbers. City directories are also frequently 
neither current nor accurate. For example, two of the authors 
of this manual drew a 1,000 person sample from a telephone 
book- Immediately after the questionnaires were mailed, 
over 150 were returned witn the U.S. Postal Service Stamp 
"Moved—No Forwarding Addressl" Typically, however, 
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populations in school related surveys v;ill be a more 
confined group, such as parents of high school students or 
home economics teachers in a specific district. Regardless 
of the size^^.^the target population must be represented as 
the sample ^x's^ drawn. 

Drawing . Once the target population is defined, one 
simple method of selecting a random sample may be conducted 
by assigning a number to each member of the population. 
Corresponding numbers are then written on slips of paper 
with one slip representing each member of the population. 
The slips of paper are then placed in a container and 
thoroughly mixed up. Slips of paper can then be drawn one- 
by-one to identify the members of the population who will be 
surveyed. 

Using random numbers . A table of random numbers or a 
computerized method may be more desirable ana systematic for 
selecting random samples for larger surveys. These methods 
are also most appropriate for smaller surveys. Remember, 
when using simple random sampling, the entire population must 
be clearly defined and it must be equally possible for each 
member to be chosen for the sample. 

Stratified Samples 

Stratified sampling is a method used to obtain a 
greater degree of representation. Rather than selecting a 
sample from the total target population, appropriate numbers 
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are drawn from homogeneous subsets of that population. In a 
study of high school students, for example, the researcher 
may first organize the population by class and draw appropriate 
numbers of freshmen, sophomores, juniors, and seniors. The 
drawing must be representative of the percentage of the total 
for each class. 

The researcher might wish to utilize an even more 
complex stratification method. In addition to stratifying 
by class, the researcher might also stratify by sex, by 
grade point average (GPA) , and so forth, in this fashion, 
the sample would contain equivalent numbers of freshmen boys 
with a 4.0 GPA, of freshmen girls with a 4.0 GPA, and so 
forth. 

The choice of stratification variables typically 
depends on what variables are available and the goal of the 
survey. Sex can often be determined from a list of names. 
Lists of teachers are commonly prepared according to school 
district, school, and perhaps subjects taught. In selecting 
stratification variables from among those available, the 
researcher should be concerned primarily that those selected 
can be represented accurately. Sex, education, and geographic 
location are frequently stratified. 

The method of stratification usually requires grouping 

the population into discrete groups based on whatever 

stratification variables are used. On the basis of the 

relative proportion of the population represented by a given 
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group / a number of elements from that group are selected-— 
either randomly or systematically. For example, if freshmen 
boys with a 4.0 GPA comprise one percent of the population, 
and a sample of 1,000 students is desired, then 10 freshman 
male students with GPA*s of 4.0 must be selected. The effect 
of stratification is to insure the proper representation of 
the entire population in the sample which will be surveyed. 

Systematic Techniques 

Systematic sampling may be less laborious than simple 
random sampling, but does have some pitfalls • From the 
entire population, each "nth" member is chosen for the sample. 
For example, if there are 1,000 members in the target popu- 
lation and 100 are to be surveyed, every 10th member is 
chosen, beginning with any number beween one and ten. The 
starting number is randomly picked from a listing of "n" 
numbers or from a table of random numbers. In the first 
example, a number would be randomly selected from numbers 
ranging from one to ten. If three were selected, then numbers 
in the sample would be 3 , 13, 23, 33, . . . until 100 subjects 
were chosen. 

To avoid biasing a sample when using the systematic 
sampling technique, the listing of the population musb not be 
cyclical. That is, it should not have any repeating pattern. 
For example, 12 teachers from 12 elementary schools are to 
be surveyed. Each school has listed their teachers by grade 
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level and each school has two classes for all grades, 1-6. 
If every 12th teacher is to be surveyed,, and one began by 
selecting the second name and chose each 12th name thereafter, 
only sixth grade teachers could conceivably be surveyed. 
For this reason, great attention must be given to the manner 
in which lists of the population are prepared to insure that 
cyclical patterns are not present. 

Sample Size 

There are no absolute standards regarding the 
percentage of persons in a population who should be surveyed. 
However, it is imperative that those chosen be sufficiently 
representative of the entire population in the variables for 
which the sample will be analyzed. No matter what the sample 
size, data must be analyzed statistically to determine the 
confidence with which generalizations can be made concerning 
the entire population. 

Table 2 provides a rough estimate of the sample size 
needed for various populations. Please remember that the 
sizes are only rough estimates and that each survey must be 
considered individually for its unique characteristics. No 
statistical inferences may be determined from the figures 
provided . 
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Table 2. Estimated Sample Size 





Population 


Sample 


Population 


Sample 


Size 


Size 


Size 


Size 


100 


79 


5,000 


357 


200 


132 


6,000 


361 


300 


168 


8 ,000 


367 


400 


196 


10,000 


370 


500 


217 


15,000 


375 


600 


234 


20,000 


377 


800 


260 


30,000 


379 


1,000 


278 


50,000 


381 


2,000 


322 


75,000 


382 


3,000 


341 


100,000 


383 



SOURCE: The National Education Association. Table 2 
is reproduced with the written permission of the NEA. 



Is the Nonrespondent a Factor ? 

A number of studies have analyzed the significance of 
nonresponses . Larry L. Leslie (1972) conducted a thorough 
study of nonresponse bias and concluded that: (1) when 
surveying populations with a common group identity (e.g., 
parents from one school, teachers in a school district), 
response differences between respondents, nonrespondents , and 
late respondents are unlikely; and (2) the most likely 
exception to the above is the case when the topic overrides 
importance to group membership, e.g., highly personal or 
sensitive areas. 

In 1960, Marjorie N. Donald found no significant 
demographic differences between respondents and nonrespondents 
in a study involving members of the Womens* League of Voters. 
Differences were found between respondents and nonrespondents 
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in the amount of involvement or interest that individuals 
had toward the research topic. Yet the obtained responses 
did not differ between the two groups. 

Table 3 illustrates the percentage of returns Donald 
received after the initial mailing and subsequent follow-^ups. 

Table 3. Percentage of Returns and Follow-up Techniques 

Mailing or Follow-up Percentage Returned 

First mailing 46.2% 

First follow-up letter 12.3% 

Second follow-up letter plus second 

copy of questionnaire 8.8% 

Telephone call plus third copy of 

the questionnaire if desired 10.1% 

Total 77.4% 

M. N. Donald, "Implications of Non-response for the 
Interpretation of Mail Questionnaire Data," Public Opinion 
Quarterly 24 (1969): 99-114. 

By observing Donald's data, it may be seen that initial 
questionnaire return rates can be greatly increased with 
additional follow-ups. 

The impact of nonresponuents can be determined by 
comparing "waves of respondents" (early, average, and slow 
returners). According to Leslie, if the population is rather 
homogeneous and if responses between waves do not differ, then 
a very high response rate is probably not necessary. However, 
it is still the responsibility of a researcher to obtain the 

largest possible percentage of returns. 
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Lewis Mandell's (197 4) example of a nonresponse bias 
involved perceived discrimination of women in employment. 
Upon review of the data, a researcher might find that the 
response rate for women is 90 percent, whereas, the rate for 
men is 50 percent. Also, a greater percentage of women 
(75%) than men (40%) felt that women are discriminated 
against in employment. With the available data, the study 
results would not reflect the feelings of the total population, 
thus a nonre&ponse bias is produced. 

Selecting the Appropriate Response Type 

Predetermining Categories 

How dc.s an investigator determine the exact type of 
question construction to be used when designing a survey? If 
respondents are forced to choose from categories which do not 
relate to their feelings, a bias is produced. When questions 
which require dichotomous answers (yes-no, agree-disagree, 
never-always) are used, all possible responses must be 
considered. For example, if a person were asked "Do you favor 
increased vocational training?" — Yes or No. A person might 
choose to omit the item since such an opinion would undoubtedly 
require qualification — e.g., increased for whom, when, where? 
However, if a question is worded precisely and there is any 
chance a respondent could not choose between dichotomous 
answers, then a continuum must be used. The five-point Likert 
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scale is the most widely used scale and the most highly 
recommended . 

A question like, "Do you have an overhead projector 

in your room at all times?" does not require a scaled choice 

of responses. Yes or No are logically the only two possible 

responses. However, a question assessing job satisfaction 

would require the use of a continuum. 

5. Check the answer that best describes your present 
job satisfaction. 



5.1 Very satisfied 

5.2 Satisfied 

5.3 Don't know 

5.4 Dissatisfied 

5.5 Very dissatisfied 



Forced response items with exhaustive categories can 

gather useful data. Again, it is important to use "other" as 

one response category if exhaustive choices cannot be 

categorized. A study might be conducted to examine vocational 

guidance resources for students. One question could be: 

6. Which of the following persons would you visit 
first for vocational guidance? (Mark only one.) 



6.1 Relative 

6.2 Parent 

6.3 Friend 

6.4 Counselor 

6.5 Principal 
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6.6 Teacher 

6.7 Other (please specify) 

A rank order format may be usee to gather this 

information. 

7. Rank the following persons in the order you v?ould 
seek vocational guidance from them. (Place a "1" 
in front of the first person you would consult, 
a "2" for the second, etc., until you have ranked 
all six or seven possibilities.) 



7 ♦I Relative 

7.2 Parent 

7 . 3 Friend 

7.4 Counselor 

7.5 Principal 

7.6 Teacher 

7.7 Other 



The rank order format, in this instance, provides the 
researcher with more information. If the researcher is 
interested only in the first preference, then the first form 
is most appropriate. However, if the average preference for 
each category contributes to the goals of the study, the 
ranking form is most appropriate. 

Using Forced Response Items 

If a forced response item is used, an investigator 
must adequately anticipate all the significant alternatives 
without overlap. Further, all items must be logically or 
consistently ordered and the wording of items must be 
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impartial and unbiased. Designing forced response categories 
that obtain accurate results requires careful consideration 
for every alternative. Often questionnaires contain alterna- 
tives which are too similar, thus unnecessarily dividing 
responses* 

The following continuum, or one very similar to it, 
is often used to determine the use of some teaching method or 
the availability of an educational product. The question 
might be: 

8. How many times do you use the overhead projector 
in your Home and Family Living classes? 



8 . 1 always 

8.2 usually 

8.3 sometimes 

8.4 infrequently 

8.5 seldom 

8 . 6 never 



Other than always and never, the spectrum of responses 
is so very close that it is nearly impossible to draw 
worthwhile or reliable conclusions. 

One better method is to place some precise or 
quantitative elements in the continuum. In the case cited 

4 

above, at least a more definitive set of conclusions could be 
inferred if the response set were: 
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9.1 every day 

9.2 at least once a week 

9.3 at least once a month 

9.4 less than once a month 

9.5 never 

Another very frequent error in question writing is 
to require a single response when .the question has two parts. 
Assume that you received a questionnaire which, in part, 
asked: 

10. To what extent do you approve of team teaching or 
flexible scheduling in your Vo-Ag classes? 

[ ] 10.1 Greatly approve 

[ ] 10.2 Approve 

[ ] 10.3 No opinion 

[ ] 10.4 Disapprove 

[ ] 10.5 Greatly disapprove 
Although the continuum is technically correct, the question 
is invalid. The respondent is asked to make a joint decision 
about two completely different concepts. There is no possible 
analysis that can identify which of the pair is being judged. 
This type of plural question is very often used. In this 
case there should be two distinct questions. Also, when a 
respondent is asked to make a value judgment of some kind 
the response choices should not be numbered. Numbers seem 
to indicate a rating scale, although you mi.ght explain the 
instrument design in a cover letter to alleviate this 
unwarranted conclusion. 
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other biases may be constructed into a survey 

instrument if the investigator does not analyze each set of 

responses for equal weighting, or in controversial issues, 

a positive, neutral, or negative position. 

For example, a survey of public opinion concerning 

the effectiveness of vocational training in high schools is 

planned. If a disproportionate number of questions are 

concerned with the positive aspects of the curriculum, 

possible negative public attitudes may not be elicited. The 

item which follows illustrates this fallacy. 

11. Concerning our high school vocational program, 

to what extent do you think it is doing a good job? 

[ ] 11.1 Doing great 

( ] 11.2 Doing an adequate job 

[ ] 11.3 Doing about as good as we can afford 

[ ] 11.4 Could be improved 

[ ] 11.5 Not doing a good job 

Note above that four of the five responses are 
positively oriented, while only one is negative. There is no 
neutral response. On any random sample the bias ought to 
show 80 percent responding favorably to this item and 
20 percent responding negatively. The above bias is a very 
f frequently observed one in response construction. 

To alleviate the positive or negative bias, as may 
be the case, a response set of five should contain the 
following elements: 
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[ ] 1. Very Positive Response 

[ ] 2. Positive Response 

[ ] 3. Neutral Response 

[ ] 4 . Negative Response 

[ ] 5. Very Negative Response 

The model above is mandatory for Likert-type response sets. 

If the researcher desires to use but three items in 

a response set, then a positive, neutral/ and negative 

spectrum should be used. Biases can be built into forced 

response items — either by purposeful design or ignorance. 

The following illustrates this point. 

12. Put a check [/ ] in the box that preceeds your 
favorite make of car. 

[ ] Ford 

[ ] Dodge 

[ ] Chevrolet Impala 
[ ] Chevrolet Caprice 
[ ] Fiat 
[ ] Toyota 

[ ] Other (please specify) 

Did you detect the two causes of bias? First, the response 

choice for Chevrolet is divided, potentially reducing its 

rating. Secondly, the omission of popular cars such as 

Volkswagen may preclude their being rated even though the 

category of "other" is offered. The above checklist would 

yield invalid results, at best, especially if several 

respondents checked more than one make of car. 
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Another source of bias is to request professional 

judgments of the general public, for example, if one were 

asked the following: 

13. I favor a 12-month school year because students 
have much more to learn than in the past. 

agree 

disagree 

do not know 

Some respondents would be in a quandry, if they 
favored a 12-month school year, but with not all students 
attending at the same time. The two notions: (1) favoring 
a 12-month school year, and (2) students have more to learn 
now than in the past, should be presented in two different 
questions. In addition, if this question is addressed to 
the general public the situations should be defined. 

To what extent is a question likely to be influenced 
by the context of the preceding one? There is a dangerous 
way of "prompting" a respondent. In a series of questions, 
a respondent may be led to a decision that would not have 
been made if a different line of questioning had been used. 
A researcher is ethically bound not to be dishonest or to 
willfully structure a questionnaire to "prove a point." 

Consider this case. If, after a number of questions 
concerning which occupations have the most respect, highest 
pay, and most satisfaction, a student is asked what he or she 
plans as a future occupation, the response will surely be 
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biased. Such a design is referred to as "prestige bias," 
People enjoy prestige, even if only in questionnaires. This 
bias can be avoided by careful placement of items throughout 
the instrument . 

Some considerations must be given to influences 
outside the survey. The timing of a survey alone, for example, 
can be a biasing factor. For example, officials wish to know 
the number of seniors who plan to apply for work at a large 
local lumber mill. A questionnaire is sent out immediately 
after a major labor dispute has occurred. Possibly, the 
survey results might be substantially different from a similar 
survey taken at a different time. 

The number of types of biases is nearly equal to the 
number of types of studies conducted. Each study may have a 
unique type of bias which must be minimized or eliminated if 
one were to arrive at valid conclusions. The ultimate end 
of a survey is to determine perceptions at some point in 
time. If the perceptions are validly sampled then a high 
degree of credibility will be gained by the researcher. And, 
the reverse is also true! 

Now, on to the conduct of the survey, which is the 
focus of Chapter 6 . 
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CHAPTER 6 



CONDUCTING THE SURVEY 

Knowing the Target Population 
Vocational and career educators are faced with the 
problem of planning, implementing, and evaluating current 
and projected programs each year. One way to obtain needed 
information to help make decisions or solve problems is to 
seek opinions from the concerned clientele. A first step is 
to identify the population group which can best provide the 
most meaningful information to help in the decision making 
processes. These important groups are called the "target 
population . " 

A target population may include more than just a 
single interest group. For example, a recent study wanted 
to assess attitudes about expanding vocational education 
sites outside of the existing school facility. Investigators 
determined that it would be important to obtain attitudes not 
only from the vocational and career instructional personnel 
of the school, but also from the involved parents and the 
students. By incorporating these segments of the educational 
community, the investigators received three different points 
of view on which to base a decision — a decision based on 
relevant and meaningful data. 
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To obtain the most complete and unbiased data, 
researchers should be sensitive to unique characteristics 
of target populations* For example, if these subgroups have 
already been "bombarded" with interviews and questionnaires, 
there may be reticence to respond to yet another survey. 

Community Resource People 

In recent years, ethnic minorities and the socio- 
economic disadvantaged populations have been priority targets 
for all sorts of researchers* After being frequently 
massaged for information, these respondents often do not 
perceive any positive result from the all too numerous 
studies* How then, can you obtain planning information from 
a unique audience? 

A few techniques have proven successful. One method 
is to enlist assistance from resource persons already in the 
community. These individuals may possibly be able to provide 
all the needed information. If they cannot provide complete 
and accurate information and a survey is necessary, the 
resource persons, because they have established rapport with 
the target group, may actually be more successful than the 
researcher in collecting the needed data. For example, the 
county cooperative extension or home demonstration agent might 
have established communication channels with a particular 
target population. It may be beneficial to the researcher to 
engage these agents by having them write a cover letter or 
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conduct the interviews* Other resource persons might be 
members of the ministry, scout leaders, local business 
managers, and employment service administrators* 

Cardozier (1971) suggests that alternative sources 
of information could be used in the event that a survey 
might not be possible. He suggests using public resources 
such as the records of public schools, census data, welfare 
offices, and chambers of commerce • In other words, collect 
data at primary record sources. These records will be 
accurate and allow a researcher to complete data collection 
in an unobtrusive manner. 

Questionnaires are most frequently used to identify 
a problem or to show that a particular or emerging need 
exists. A questionnaire oriented to a group of people^ can 
be a powerful instrument, but so can a respected professional. 
A resource person knowledgeable of the field can be very 
helpful or persuasive to others. 

Resource persons must be given specific, unbiased 
questions to ask their contacts. They must also understand 
that the collection of data requires accurate reporting. 
Most likely, when you enlist the aid of community resource 
persons, you will find that a considerable amount of training 
must also be undertaken — including a few simulated interviews 
to insure quality work. 
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Protection of Participants 
After the target population has been defined, 
investigators must realize that recent federal legislation 
has placed limitations on methodologies that can be used when 
conducting research on human subjects, i.e., the participants. 

Individuals who write proposals concerning behavioral 
research which will be submitted to the Department of Health, 
Education, and Welfare (D/HEW) , should be aware of recent 
legislation on the protection of human rights (Public 
Law 93-348, 1974, short title): "The National Research 
Service Award Act of 1974," Title II, Protection of Human 
Subjects of Biomedical and Behavioral Research," establishes 
procedures that must be followed when any person is involved 
as a subject in a behavioral studies program funded by D/HEW 
or any other federal agency! 

The law is lengthy and technical. For our purposes, 
it is necessary to know that a National Commission for the 
Protection of Human Subjects of Biomedical and Behavioral 
Research has been established. Its duties include establishing 
guidelines for biomedical and behavioral research to protect 
human rights. The Commission's responsibility entails 
developing methods for evaluating and monitoring the perfor- 
mance of institutional review boards. Also, it investigates 
biomedical and behavioral research conducted or supported 
under HEW programs involving children, prisoners, and 

institutionalized mentally infirm. 
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The Assistant Secretary for Planning and Evaluation 



(D/HEW) established rules for the protection of the privacy 
of persons who are the subjects of projects sponsored by 
D/HEW. The following excerpt from the Federal Register is 
very explicit and illustrates the kinds of constraints placed 
on human research, of which vocational education surveys 
would qualify. 

(c) Protection of privacy. (1) No project supported 
under this part may involve the use of a data collection 
instrument which constitute invasions of personal 
P3:ivacy through inquiries regarding such matters as 
religion, sex, race, or politics. (2) A grantee which 
proposes to use a data collection instrument shall set 
forth in the grant application an explanation of the 
safeguards which will be used to restrict the use and 
disclosure of information so obtained to purposes 
directly connected with the project, including provisions 
for the destruction of such instruments where no longer 
needed for the purposes of the project. 

(d) Clearance of instruments. (1) Grantees will not 

be required to submit data-collecting instruments to the 
Assistant Secretary or obtain the Assistant Secretary's 
approval for the use of these instruments, except where 
the notification of grant award specifically so provides. 
(2) If a grantee is required under paragraph (d) (1) of 
this section to submit data-collection instruments for 
the approval of the Assistant Secretary or if a grantee 
wishes the Assistant Secretary to review a data-collection 
instrument, the grantee shall submit seven copies of the 
document to the Assistant Secretary along with seven 
copies of the Office of Management and Budget's standard 
form No. 83 and seven copies of the Supporting Statement 
as required in the "Instructions for Requesting 0MB 
Approval under the Federal Reports Act (Standard form 
No. 83A) . 

Of importance to all researchers is that the data 
gathering instrument does not seek sensitive information. In 
some cases, your instrument must be screened by a Human 
Subject Review Board. 
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Do not hesitate to get clarification on how you may 
use human subjects in federally funded projects. Guidelines 
may be obtained by contacting any of the regional D/HEW 
Field Offices. In the Northwest, for example, contact: 

United States Government 

Department of Health, Education, and Welfare 
1321 Second Avenue 
Seattle, WA 98101 

Inclusionary Language 

Another aspect to consider when writing requests for 

funds, data gathering instruments, cover letters, or final 

reports is to use inclusionary language. Inclusionary 

language helps to remove sex bias in dialogue and is important 

to both sexes when describing vocational or career roles. 

The following statements are offered as comparative samples. 

Example 1 . "After a student completes the welding 
course, he will be able to use three different styles 
of beading." 

The implication of this statement is that females are 
excluded from taking welding courses. The statement could 
have been improved if it had been written with inclusionary 
language : 

Example 2 . "Upon completion of the welding courses, 
students will be able to use three different styles of 
beading . " 

Example 3 . "Upon completion of the Family Living course, 
she will~be able to plan a weekly budget." 

Again, this implies that males have no desire to be 

involved with a course in Family Living. The more appropriate 

writing would be: 
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Example 4 , "Upon completion of the Family Living 
course, the participant will be able to plan a weekly 
budget. " 

Other examples . When writing cover letters or 
follow-up letters use alternatives for the traditional 
"Dear Sir" when the sex of the addressee is unknown. Other 
salutations that could be used include: Dear "Supervisor," 
"Director," "Owner," "Faculty Member," or "To Whom it May 
Concern." There are also alternatives that may be used when 
describing various vocations such as "letter carrier" instead 
of "mail man," "human resources" instead of "manpower," and 
"newscaster" instead of "newsman." 

By using inclusionary language, educators are 
broadening the awareness of the world of work for all concerned. 
Stereotypical vocational role expectations for the sexes have 
become a major concern of our society. The writer of survey 
instruments must also become sensitized to these societal 
concerns. 

Communicating with the Target Population 

Written Correspondence 

Once the target population has been selected, the 

next task is to insure that the participants receive and 

respond to the survey instrument. An effective cover letter 

explaining the reason for the study helps in obtaining a good 

return of the survey instrument. The cover letter must tell 

why it is important for the respondent to give personal 
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attention to completing the instriiment • A well-written letter 
is a persuasive and motivating device. 

The following points should be addressed when designing 
a cover letter; 

1, State the purpose of the study clearly • 

2, Explain how the data will be used, 

3, Explain the value of the study, 

4, Identify the sponsoring agency or institution, 

5, Include the investigator's name and the name of 
the study's sponsor, 

6, Give explicit directions for completing the 
questionnaire, 

7, Provide a reasonable deadline for returning the 
questionnaire , 

8, Assure the respondent that the data will be handled 
confidentially, 

9, Ask if a copy of the results of the study are 
desired by the respondent, 

10, Provide a self-addressed, stamped envelope, 

11, Date the letter, 

12, Sign the letter personally, 

13, Use an original or an extremely high quality copy, 
A sample cover letter that contains all the poxnts 

that have been mentioned above follows as one model. 



85 



ERiC Hfi 



San^ple Cover Letter 



Rainier Public Schools 
Last Chance Gorge, Washington 98999 



Date 



Dear Vocational Educator: 

The Office of Evaluation and Research of the Rainier 
Public Schools, in cooperation with the Office of the 
Superintendent of Public Instruction, is conducting a study 
to assess the supply and demand of vocational teachers in 
this state. This information will be used for future planning 
in vocational certification program offerings provided by 
state colleges and universities. 

Enclosed is a questionnaire designed to complete the 
study. Your opinion would be greatly appreciated. Just mark 
the appropriate response for each item. If you would like 
to include comments of your own please write them in the 
Remarks section. All responses will be handled confidentially. 

We hope to complete the study by (date) and your 
cooperation will help us a great deal. If you want a copy 
of the completed study please so indicate on the questionnaire. 

Sincerely yours. 



Dr. Robyn A. Bird 
Project Director 



Ms. Dianne L. Hunt 
Project Assistant 

RAB: ts 

Enclosure: Questionnaire 

Return envelope 
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Follow-ups . The investigator must usually conduct 
a follow-up after the initial mailing if the questionnaire 
return rate has not been adequate • There are several methods 
that can be used. One technique is to send a second letter 
which "gently" requests cooperation in returning the 
questionnaire. Another method is to send a second question- 
naire and return envelope with the second letter. 

Postcards may also be used in the follow-up. With 

tight budgets, postcards are less expensive. A suggested 

postcard note follows. 

Dear Vocational Educator: 

Two weeks ago you received a questionnaire 
concerning vocational programs in your school. 
This is just a reminder to ask your help in completing 
and returning that questionnaire. If you have already 
done so, thank you. If not, your reply is needed to 
help in assessing vocational teacher supply and demand 
in the state. 

Thank you for your cooperation. 

Dr. Robyn A. Bird 
Project Director 

If you have adequate budget, use a telephone contact 

with tardy respondents. 

Consideration for the Respondents 

As vocational and career educators the problem of 
finding time to complete questionnaires is sometimes difficult. 
Therefore, when you send out questionnaires, you can understand 
some of the problems of the recipient. 
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Consider the limitations of the questionnaire when 
deciding whether or not to use one. These limitations are 
also useful as guidelines in preparing a survey. Here are 
some suggestions that should be considered when attempting 
to get maximum response* 

1. Is the entire questionnaire concise enough so that 
the respondent will not have to use time "guessing" 
what is required? 

2. Will the subject be interesting to the respondent? 
If the subject of the investigation does not have 
any general interest, completed returns will be 
practically nil. 

3. Is the questionnaire verbally designed (vocabulary, 
technical jargon) for a particular target population? 
You must know the language and vocabulary 
limitations of the clientele and create an instrument 
that can be understood by them. 

4. Have the questions been printed with crisp, clean 
type? The physical appearance of the questionnaire 
can affect return rate. If the instrument has a 
messy appearance, it implies that the investigator 
will handle return data in an unprofessional manner. 

5. Does the questionnaire look professional? Are the 
questions presented in an attractive, uncluttered 
way? An orderly presentation of items helps the 
respondent to reply more efficiently. 

6. Is the questionnaire easy to handle? In other words, 
is the instrument void of loose inserts, fold-outs, 
and requirements to cross reference particular items? 

7. Are the directions for completing the questionnaire 
stated clearly? If a recipient is not afforded 
appropriate instruction on "how and what" to do, a 
large return cannot be expected. 

From the respondent's point of view, the ease with 
which an instrument may be completed is very important. 
Remember, you may have gone through the same experience, so 
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be considerate of your participants. Consequently the 
investigation will be beneficial ♦ 

The "tips" presented in this chapter should aid 
in the overall conduct of a survey • We desire to imply that 
conducting a survey is a rigorous and serious effort — not 
something that can be hastily constructed and distributed. 
Of course, once the data are collected, then what? That 
question is answered in Chapter ?• 
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CHAPTER 7 



TABULATION OF DATA 

Methods of Tabulation 

Designing a survey requires that a plan be specified 
for tabulating the returned data» In most cases, a response 
count of each category is required ♦ The purposes of survey 
research are to determine general characteristics and 
opinions of a sample population; all data must be summarized 
to achieve these objectives ♦ 

There are two general methods for tabulating data. 
The first method, hand tabulation, is less efficient, more 
time consuming, yet practical for a limited number of 
questionnaire items or a small sample size^ The second 
method, a readily available alternative, is electronic data 
processing ♦ This method is invariably used for larger sample 
sizes and for most statistical analyses ♦ 

Hand Tabulations 

Tabulation of data by hand is a technique which is 

usually efficient for small sample sizes or where there are 

relatively few questionnaire items ♦ No electronic data 

processing equipment or special material is necessary. The 

method consists of simply counting the number of responses 
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for each category of each questionnaire item* Efficient 
hand tabulation requires planning, consistent organization, 
and neatness. Below are two sample questions followed by a 
layout for tabulating their results. 

1. I am 

1.1 male 

1.2 female 

2 . I am 

2.1 Under 20 years of age 

2.2 20-29 

2.3 30-39 

2.4 40-49 

2.5 50 years or over 



Table 4. Tabulation Form for Survey Items 



Variables 


Categories 


Totals 


Sex 


Male III 


10 


Female |>U i | 


Age 


Under 20 II 


10 


20-29 III 


30-39 1 


40-49 ill 


50 and over | 


No response 
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For illustration purposes in Table 4 , assume that 
10 respondents marked the above two questions. Three of the 
respondents were male and seven were female* Two were under 
20, three were between 20-29, one was between ages 30-39, 
three were 40-49, and one was 50 and over. The column 
labeled "Totals" indicates the number of respondents who 
marked each questionnaire item. The "No Response" variable 
indicates how many respondents neglected to mark each 
questionnaire item. These counts are important as a check 
against tabulation error and for further analysis involving 
percentages and means. The tabulation procedure requires 
examining each questionnaire returned, noting which category 
represents that particular respondent, and tallying the 
response appropriately. It is a prudent practice to place a 
mark on each questionnaire which has been tabulated. In this 
manner, you will know which instruments have been counted and 
which have not. 

Frequently, researchers are interested in determining 
relationships between the response categories of two or more 
questionnaire items, for example, the number of males and 
females in each age category. Data can be specifically 
arranged to indicate the frequency of response for each of 
these subcategories. Table 5 illustrates an efficient layout 
for tallying the relationship between two questionnaire items. 
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Table 5. Tabulation of Two Items 



Age 


Sex 




No Response 


Male 


Female 


Totals 


No response 














1 




2 


20-29 






III 


3 


30-39 








1 


40-49 




II 




3 


50 and over 








1 


Totals 




3 




10 



As you examine each questionnaire, note whether the respondent 
is male or female, and the appropriate age category. Mark 
the tabulation sheet in the appropriate combination sex and 
age cell. Note that there is only one cell appropriate to 
both of these categories. The cells labeled "Totals" in 
Table 5 provide a separate count of the response categories 
of each questionnaire item. In this example there is a 
separate count of males, females, and each age category. For 
example, in Table 5 there are two respondents under 20, three 
between the ages of 20-29, one between 30-39, three between 
40-49, and one 50 and over. A total of 10 respondents marked 
the age item. 

Compare Tables 4 and 5 and note that it would be 
redundant to tabulate the data in both forms. Table 5 yields 
the same information as Table 4, in addition Table 5 indicates 

93 



ERLC 



104 



the relationship between the subcategories of two question- 
naire items. If relationships between questionnaire items 
are important, these same items do not necessitate individual 
tabulation. By grouping data according to your interests, 
more efficient and detailed displays of data can be obtained. 
Before you begin tabulating, organize a tabulation form for 
the entire questionnaire to assure efficiency and lack of 
repetition. 

When ranking is required, each category of the 
question must be tally ed for each possible preference. Below, 
in Table 6, a first preference choice for Metal Shop was 
selected by six respondents. Woodworking received two first 
preference choices, while Airframe received three. The 
number of respondents who selected each of the classes for 
second and third choices is also tabulated. 



Table 6. Tabulation of Ranked Items 



Preferences 


Classes 


Metal Shop 


Woodworking 


Airframe 


First 


nil 1 


II 


III 


Second 


II 


nil 1 


III 


Third 


III 


III 


Mil 


No response 
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Electronic Data Processing 

Data are most frequently tabulated and analyzed by 
electronic equipment. Three standard pieces of machinery 
are the key punch, the card sorter, and the computer or 
printer. These items of equipment are accessible in most 
towns. Vocational schools and community colleges typically 
have a key puncher and a card sorter. State colleges and 
universities invariably possess the equipment, and will 
process outside data, usually charging only for personnel. 
Private businesses charge slightly higher fees for the same 
services . 

Tabulation and analysis needs for the questionnaire 
should be planned in accordance with the budget before the 
research begins. If the use of electronic data processing 
is planned, the instrument should be examined by a data 
processing specialist before the questionnaire is printed. 
It has been very sad for many researchers to determine, after 
data have been collected, that the items are not transferable 
to data cards. Thus, protect your study by seeking expert 
advice in the early planning stages. 

Data are transferred from each questionnaire to 

electronic data processing (EDP) cards, which are a medium 

for systematic representation. The illustrated EDP card 

(Fig. 1), also called an "IBM card," is divided into 

80 vertical columns, with 10 numbered spaces in each vertical 

column, zero through nine. Data are made machine readable 
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by assigning one or more specified vertical columns of the 

card to a particular questionnaire item and assigning spaces 

within that column to the various possible response 

categories. It is efficient when transferring the data to 

the card, if the questionnaire item number is coded to 

correspond directly to the vertical column number of the 

EDP card. For example, if the first item of a questionnaire 

requests the respondent's age and there are fewer than 

10 response possibilities, this item can be represented in 

vertical column one. If the age data are divided into five 

categories, six of the 10 numbered spaces of the column will 

be needed to represent the respondent's age. Row zero is 

usually reserved to indicate "no response." The response 

category coded (1) should be represented in row one for easy 

transfer of data. By numbering the specific response 

categories to coincide with the row numbers, an easy coding 

device is automatically established with your questionnaire. 

A single EDP card normally accommodates one respondent. 

If more than 80 vertical columns are needed to record the 

responses of one individual it is necessary to use two or 

more cards. The last few vertical columns should be used 

to mark the card for identification. As a check against 

transfer errors, each respondent should be assigned an 

identification number to be written on both the original 

questionnaire and the appropriate EDP card. If more than 

one card per respondent is necessary, the card number, 1, 2, 
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3, etc., should be indicated on the EDP card in a specified 
column* Figure 2 has identification numbers marked in the 
last columns. It is card number 2 (indicated in column 80) 
representing respondent number 25 (indicated in columns 78- 
79) . 

Preceding 

Preceding a questionnaire allows one to key punch or 
hand tabulate directly from the questionnaire. Preceding 
forced response data involves assigning questionnaire items 
and response categories to EDP card columns and rows in 
advance of printing the questionnaire. Notations are then 
made on the questionnaire that indicate these assignments. 
There are two steps involved in preceding a questionnaire. 

Step one . Card and column assignments should be 

indicated for each question. Typically, the column number 

is shown in parentheses in the left margin beside each item. 

This should be done inconspicuously so as not to distract the 

respondent. Otherwise, explain the use of these numbers to 

the respondent in the introduction or cover letter. The 

illustration below shows a preceded questionnaire item. 

[3] 3. Which of the following best compares this year's 

(1974-75) vocational agriculture curriculum to next 
year's (1975-76) planned vocational agriculture 
curriculum? 

(1) Fewer classes will be offered next year 

(2) Same for both years 

(3) More classes will, be offered next year 
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In this example, column three of the EDP card will be used 
to represent question three. It is more expedient when the 
questionnaire item number and the column number are the 
same; but this is not always possible with ranked or 
checklist items. 

Another format may be used for the same question: 
The questions and responses remain the same, but the response 
items have a numerical system that identifies both the column 
and the row. For example: 

3. Which of the following best compares this year's 
(1974-75) vocational agriculture curriculum to 
next year's (1975-76) planned vocational agriculture 
curriculum? 

3.1 Fewer classes will be offered next year 

3.2 Same for both years 

3.3 More classes will be offered next year 

Or the numbers could follow the blank: 

3.1 Fewer classes will be offered next year 

3.2 Same for both years 

3.3 More classes will be offered next year 

The examples above quickly show that column three will use 
three rows, one, two, and three. Again, nonresponses would 
automatically be marked in row zero. Either system is 
efficient and saves valuable time in transferring data. 

Step two . The second task in preceding involves row 
assignments within a given column. If there are nine or 
fewer response categories to an item, assign the categories 
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to rows Reserve row "zero" to represent "no response." 

Only one space in each column should be punched. In the case 
of ranked and checklist items, where the respondent is 
required to respond to more than one category, each category 
must be represented by a separate column on the data card . 
The ranking assigned by the respondent for a particular 
response category should be punched in the row which 
corresponds to the ranked number, in the column which 
represents that response category. In sximmary, ranked or 
checklist items require more columns on the EDP card than do 
nominal/ ordinal, or interval items whose categories are 
mutually exclusive. 

Keep a code book or log which describes the locations 
of questionnaire items and response categories on EDP cards- 
In simple surveys, it is sufficient to list these assignments 
on a blank copy of the questionnaire. 

If a preceded questionnaire contains proper indications 
of columns and punch assignments, direct keypunching from the 
questionnaire to the EDP card is possible. To accomplish 
this, all questionnaires should be first edited to clarify 
or assign "zeros" to any unclear responses. Those questions 
that are unanswered or are incorrectly completed should be 
automatically assigned a zero. 

The layout of a questionnaire is extremely important 
in aiding direct punching. Question items and response 
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categories should be located in distinct patterns for ease 

of reading and transferring of responses to EDP cards. 

There are some instances in which precoding of 

categories is not recommended. When the respondent is 

requested to give an opinion, such as is typical with Likert 

items, numerical codes assigned to response categories tend 

to "weight" the study. Consider the following illustration 

where item 4 might ask: 

4. I would encourage my child/children to participate 
in a program that could provide skills to get a 
job. 

(1) Strongly Agree 

(2) Agree 

(3) No opinion 

(4) Disagree 

(5) Strongly Disagree 

The numbers in this case may tend to influence the respondent' 
choice. A respondent may associate a number one with the 
"best" or a number five with the "most." A factor which may 
preclude precoding interval items is the frequent use of 
reversed questions with the Likert scale. It may be 
confusing to the respondent if "Strongly Agree" were coded 
five on one question, and coded one on another question. 

When the response categories of some items cannot be 
preceded, it is necessary to prepare data transfer sheets for 
keypunching. A set of data sheets is illustrated below. 
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Table 7. Representative Data Response Models 



Blocked 
Column- 
Question 
Items 


Open 
. Numbered 
System 


Closed 
Pre-coded 
System 


Open 
Pre-coded 
System 


Parentheses 
Pre-coded 
System 


[1] 


1 . 










(1) 


1.1 


1.1 


( ) 1.1 




(2) 


1.2 


1.2 


( ) 1.2 




(3) 


1.3 


1.3 


( ) 1.3 


[2] 


2. 










(1) 


2.1 


2.1 


( ) 2.1 




(2) 


2.2 


2.2 


( ) 2.2 




(3) 


2.3 


2.3 


( ) 2.3 



Each respondent is represented by a separate data 
sheet. The key puncher simply transfers the data from the 
code sheet to an EDP card. The data sheet must be consistent 
in organization. The keypuncher should never have to "guess" 
the appropriate response nor should the keypuncher be required 
to match codes with responses. That is the researcher's 
responsibility! 

A punched deck of EDP cards is ready for virtually 
any type of analysis. The deck can be fed through a card 
sorter, which simply counts the number of respondents who 
marked each category. The card sorter can also be used to 
separate cards according to specified categories / and then 
to subdivide the categories according to another item. For 
example, the cards may be separated according to sex. In 
this case, there would be two decks; one representing males 
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and the other representing females. The researcher may 
then want to subdivide according to age. Each deck would be 
fed into the card sorter, one at a time, so that males would 
be divided according to age and females would be divided 
according to age. See Pig. 3. 



Entire Deck 




Males 




Females 




Under 20 
20-29 
30-39 
40-49 

50 and over 

Under 20 
20-29 
30-39 
40-49 

50 and over 



Fig. 3. — Subdivision of Two Questionnaire Items 

The card sorter does the same type of analysis that can be 
done using hand tabulation. Yet with the card sorter, subsort 
or category sorts can be accomplished for an unlimited set 
of arrays and all within a relatively few minutes. Such 
output allows the researcher to spend more time interpreting 
the data, rather than simply tabulating it. 

Punched EDP cards are ready to be fed into a computer 
when accompanied by a program which specifies a certain 
analysis. There are numerous preprinted programs (commonly 
called "canned programs") that usually require few additional 
computer instruction cards. Programs can be selected that 
range from simple means and percentages to complex statistical 
analyses. 
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The advantages of electronic data processing are 
numerous. The researcher's time is greatly economized by 
the speed with which electronic equipment analyzes data. 
Accuracy is improved because humans are involved in few 
steps. There is less opportunity for human error. Also, a 
deck of EDP cards can be used again and again for different 
analyses by selecting different programs. Although organizing 
the survey may take somewhat longer when using electronic 
data processing, the pay-off to the investigation can be very 
substantial because greater analyses of the data can be 
conducted. In some cases, percentages are very adequate. In 
other situations, you will want to use some inferential 
statistics. Chapter 8 provides detail about two commonly 
used treatments when using questionnaire data. 
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CHAPTER 8 



STATISTICAL ANALYSIS OF DATA 

Descriptive Techniques 
Statistical analysis refers to those methods which 
may be used to organize and interpret collected data^ A 
few of the common procedures used to summarize and describe 
sets of data are included in this section ♦ 

Respondent Counting 

The preliminary step in analyzing data is usually 
respondent counting ♦ Either of two methods are used: hand 
tabulations or electronic data processing tabulations ♦ When 
working with nominal, interval, checklist, and some types of 
ordinal data, respondent counting is a very straightforward 
procedure ♦ For review, consider the following questionnaire 
J.tem. 

1» Do you plan to attend a community college when you 
graduate from high school? 

(1) Definitely yes 

J (2) Probably yes 

(3) Undecided 

(4) Probably not 

(5) Definitely not 
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Respondent counting for item number one above would involve 
counting all who marked "Definitely yes," "Probably yes/* 
"Undecided/* "Probably not/* and "Definitely not," Those who 
did not respond to the question should be counted under a 
category entitled "no response." 

A word of caution about the latter group — nonrespondents. 
It has been observed that some people simply do not complete 
selected items or sections of an instrument. In some surveys 
this may be a substantial number. Therefore, in order for 
the total tabulation of each item to equal the number of 
returned instruments it is essential to show the number and 
percentage of those who did not respond. And, no inferences 
or speculations are allowed as to why! You can only report 
that "x" number and "x" percent of the surveyed group did not 
respond to item "2." Often nonresponses are the problem of 
a poorly worded or confusing question or item. If more 
sampling is planned, it behooves the investigator to analyze 
or seek external critiques of those "nonresponse" items, so 
that they may be rectified for future surveys. 

In contrast, analyzing ranked data requires counting 
the number of respondents who selected each category as a 
first preference / a second preference, and so forth. For 
example / the following questionnaire item elicits ranked data. 
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2. Rank each class title according to your interest* 
Use number 1 for that which interests you most, 
to number 4, that which interests you least. Use 
each rank one time only. 

Air frame 

Engine repair 

Metal shop 

Woodworking 

In each category of the above item, respondent counting 
involves tabulating the number who selected the category as 
first preference, second preference, third preference, and 
fourth preference. The technique will be amplified later 
in this chapter. 

Percentages . Respondent counting provides a summary 
of the tabulated frequency for which each category is marked. 
Frequency data can be converted to percentages indicating the 
number of respondents who marked a particular category in 
relationship to the total number of respondents. Percentages 
are usually calculated for nominal, some ordinal, interval, 
and checklist items. Table 8, for example, shows a method 
of displaying data concerning the distribution and return 
of a questionnaire. 

Caution should be noted here. The final column 
cannot be averaged to obtain 81.5 percent. The total 
percentage figure must be 155/190 x 100. If you average the 
percentages there will be great error. This point is illus- 
trated since some researchers mistakenly "average 
percentages. " 
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Table 8* Data Concerning Distribution and Return of the 

Questionnaire 



Group Polled 



Number Number Percentage 

Sent Returned Return 



Private high schools 19 16 84 

First class high schools 10 ' 10 100 

Second class high 

schools 74 58 78 

Third class high schools 87 71 82 



Totals 190 155 81.5 



Table 9 is a simulated study which is used to illus- 
trate another precaution. Examine the data and the various 
responses. When using percentages, precaution must be taken 
to describe survey respondents so as not to imply that 
4.8 percent of all home economics teachers have one teaching 
preparation per day. Any percentage figure is derived for 
a sample of respondents who participated in the survey. In 
this example, the report would correctly read: "That 
4.8 percent of the survey respondents reported having one 
preparation per day in Home Economics." 

By reporting percentages alone, the conclusions are 
usually misleading. For example, three out of ten respondents 
yield a 30% return; as does 300 out of 1,000. However, 
because the total number of respondents is much larger, the 
latter 30% might be more representative of the community as 
a whole than would be a figure of 30% which represents three out 
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of ten respondents. Obviously/ stating sample size is 
important when using percentages to describe data. We 
recommend that the number always be given with the 
percentages . 

Means. Likert scale and ranked data are frequently 

summarized by determining the average mean scores as a 

measure of the central tendency. The mean, or average, is 

computed by adding a list of scores and then dividing the 

sum by the number of scores. For example, the question 

below might be asked of 1,000 students and then tabulated. 

3. The high school typing courses adequately prepared 
you for your current stenographic position. 

Strongly agree 

Agree 

No opinion 

Disagree 

Strongly disagree 



The tabulation would be reflected in Table 10. 

Table 10. Tabulation of Question Three (Simulated) 

Response Item Number Percentage of Total 

Strongly agree 650 65 

Agree 250 25 

No opinion 30 3 

Disagree 50 5 

Strongly disagree 10 1 

No response 10 1 

Totals 1,000 100 
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To convert the number who responded to a mean or 
average, a numerical code would be provided for each 
category. The code is listed in Table 11. 



Table 11. Codes to Convert Scaled Items to Means 

Scale Item Code Weight 

Strongly agree 1 

Agree 2 

No opinion 3 

Disagree 4 

Strongly disagree 5 

No response 0 

In this example a code of "1" represents "Strongly agree" 
and "5" represents "Strongly disagree." The next step in 
determining the mean response is to multiply the code 
given to each category by the number of respondents who 
marked that category. Observe Table 12 for the model. 

Table 12. Preliminary Calculation for Mean Score of Likert 

Items 

Multiply ,^^g^g,gts 

1 X 650 = 650 

2 X 250 = 500 

3 X 30 = 90 

4 X 50 = 200 

5 X 10 = 50 
0 X 10 = 0 

Totals 1,000 1,490 
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In the above example, the computed mean score is 1.49, 
rounded to 1.5, and when reconverted falls between "Strongly 
agree" and "Agree" — a very positive response for the group. 

The algebraic formula used to determine the mean is: 

^ " IT 

when X = the mean 

ZX = the sum of the scores 

N = the total number of scores 

We have determined that ZX = 1,490, and that N = 1,000, 

therefore X = ^^^qqq = 1.49. 

The methods illustrated in Tables 10, 11, and 12 are 

very commonly used to determine the central tendency 

(arithmetic mean) for groups. The computation of this 

statistic is useful when interpretations are attached to data 

by the researcher! However, a few extremely high or low 

scores can make the mean not as typical of the central 

tendency as the median. 

Some ranked items are also commonly analyzed by 

means, as in the following illustration. Assume that 

100 students were asked to respond to item four below. 

4. Rank these class titles according to your interest, with 
number 1 being that which interests you most to number 4, 
that which interests you least. Use each ranking number 
only one time . 

Air frame 

Engine repair 

Metal shop 

Woodworking 

(Note: To avoid possible placement bias, these 
subjects are arranged alphabetically.) 
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Table 13 presents the hypothetical results of the 

survey* 



Table 13* Tabulation of Ranked Items (Simulated) 



Ranked 



Number of Respective Responses 



Preference 


Air 
Frame 


Engine 
Repair 


Metal 
Shop 


Woodworking 


First 


10 


20 


30 


30 


Second 


30 


40 


30 


10 


Third 


10 


10 


30 


40 


Fourth 


50 


30 


10 


20 


Totals 


100 


100 


100 


100 



To compute the average preference for each category 
a separate mean score must be calculated. For example, v;e 
assign a "weight" of 4 to each first place preference, 3 to 
each second/ 2 to each third, and 1 to each fourth. Table 14 
illustrates this method for Air Frame only. 



Table 14. 


Calculation 


of Mean Preference 


for Air 




Frame 


Preference 




Preference 


We ight 


Number of 


Mean 




Responses 


Product 


First 


4 


10 


40 


Second 


3 


30 


90 


Third 


2 


10 


20 


Fourth 


1 


50 


50 


Totals 




100 


200 
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The product of all weightings is 200. By dividing the product 

by the number of respondents (100) , the mean (200/100) is 

2*0 for Air Frame. This is the assigned weight for third place 

preferences. 

Using the same formula above, we calculated the mean 
preferences for each as follows: 

Air frame 2.0 

Engine repair 2.5 

Metal shop 2.8 

Woodworking 2 . 5 
We would conclude that on a mean preference, Metal shop is 
the average first preference choice. But, note that Engine 
repair and Woodworking each received an identical mean ranking 
of 2.5. Thus, other criteria could be used to determine the 
second and third preferences. Subjectivity often enters into 
the evaluation of even objectively derived data. 

Statistical Tests 
For the most part, percentages and means are adequate 
analytical methods. Yet, if an investigator were to determine 
whether differences between groups are due to chance, then 
either parametric or nonparametric tests must be employed. 
For example, if a certain question or variable were answered 
by male and female respondents in different proportions, a 
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statistical test should be computed to determine if a 
statistically significant difference exists between response 
patterns. 

In survey research, nonparametric methods are usually 
used. This condition is emphasized because one assumption 
which must be met for parametric tests is that the sample is 
drawn from a normally distributed population . In most 
surveys this basic assumption cannot be made. The use of 
powerful parametric statistics such as t and F tests or 
analysis of variance, is usually inappropriate. The second 
assumption needed for parametric statistics is homogeneity 
of variance. If a number of means are computed for a number 
of differently drawn samples, the means will surely differ 
but the variance of the means should be similar. If the 
variance of means is different then the assumption of 
homogeneity cannot be met. Again, in selecting samples for 
surveys, there are frequently wide differences in the variance. 

Continuity and equal intervals of measure comprise 
the third basic assumption to validate use of parametric 
methods. The measures to be analyzed are continuous, having 
equal intervals. The counting numbers, 1, 2, 3, . . . meet 
the third assumption. Most scales, questions, items, or 
rank ordered data used in questionnaires do not meet the 
third assumption. 
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The statistical methods that can be employed when 
any of the above assumptions cannot be met are called 
nonparametric tests. These include: 

1. The Sign Test 

2. Binomial Test 
• 3* Chi Square 

4* Rank Order Coefficient of Correlation 

5. Nonparametric Analysis of Variance 

The use of nonparametric tests is appropriate when: 

1* The assumption of normality about the population 
is suspect. 

2. The data are expressed in ranks. 

A discussion of the above five popular nonparametric 

tests would require a lengthy and detailed treatment. We 

refer the reader to three excellent sources. 

1. James L. Bruning and B. L. KintZ/ Computational 
Handbook of Statistics (Glenview, 111. : Scott , 
Foresman and Company^ 1968). 

Bruning and Kintz provide detailed explanations of 
numerous parametric and nonparametric methods. All examples 
are illustrated and clearly described. It makes statistics 
interesting/ understandable, and enjoyable. 

2. Fred N. Kerlinger, Foundations of Behavioral 
Research (New York: Holt/ Rinehart and Winston, 
Inc. / 1966) . 

Kerlinger provides a detailed discussion concerning 
many techniques related to research and statistical testing. 
He emphasizes the logic of scientific research. 

117 




128 



3. Gilbert Sax^ Empirical Foundations of Educational 
Research (Englewood Cliffs, N.J.: Prentice-Hall , 
Inc., 1968). 

Sax presents detailed chapters about questionnaire 
and interview techniques. His book emphasizes the basic 
foundations of research and also describes experimental 
designs by which to conduct and analyze different types of 
research. 

Use of Statistical Tests 
Statistical treatments, such as the Chi Square and 
Spearman's "rho" (rank order), are used to compare observations 
and to determine the strength of relationship between response 
categories of two or more questionnaire items. In many cases 
vocational educators will not use any statistical tests. The 
data will probably be reported in numbers or percentages. 
Such reporting is often adequate. Yet, when groups are being 
compared, appropriate statistical tests can strengthen the 
conclusions. 

Chi Square 

2 

Chi Square (x ) treatment in contingency tables is a 
commonly used nonparametric test for significance. Two basic 
assumptions must be met to use the test: (1) there must be 
a dichotomy or clear division of categories and (2) there 
must be a continuum of responses — at least two choices or a 
Likert-type scale. 
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In most cases an investigator will desire to test if 

the responses on selected questions are related* By computing 

the Chi Square contingency coefficient this determination can 

be made. Consider the following simulated item. Assume that 

a survey was conducted among school patrons to determine the 

feasibility of expanding vocational subjects into a svuraner 

session* The questionnaire would ask, among others: 

5* To what extent do you favor offering high school 
vocational subjects during the summer months 
for high school aged youth in our school district? 

I strongly favor 

I favor 

I have no opinion 

I oppose 

I strongly oppose 

Perhaps the investigator is interested in determining 
how various aged patrons responded to this general policy 
question* Perhaps a division at age 50 is made. The data 
would be subdivided into .all respondents 49 years old and 
younger as one category with the second category being all 
respondents 50 years old or over. Table 14 illustrates a 
model for these data. 

Table 15 is organized in a contingency table format. 
The dichotomy is the two distinct age groups. The continuum 
is the five-item Likert-type scale. By referring to a 
statistics textbook the techniques by which to compute the 
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Table 15. Responses to Question Number 5* 



Age 
Groups 



Response Category 



Strongly 
Favor 



Favor _vr..„ Oppose Strongly ^^^^^ 



No 
Opinion 



Oppose 



49 and 
under 

50 and 
over 



Totals 



Nn 



N. 



N. 



N, 



N. 



N. 



N. 



N 



10 



N 



11 



N 



13 



N 



14 



N 



15 



N 



16 



N 



17 



N, 



N 



12 



N 



18 



*The "N" represents the absolute numbers or tabulated 
responses for each category. 



X would be illustrated, e.g., Bruning and Kintz. Using the 

Chi Square test the investigator could report that: 

The response patterns on the general policy question 
of offering summer session high school vocational 
courses had a response pattern statistically different 
[or not] at the .01 level [or other selected level] 
between those respondents 49 years of age and under 
and those 50 years old and over. 

Observe how the previous statement is very specific 

and operationally defined. A reader of that finding would 

knovj precisely what was reported. This specificity of 

reporting results is essential so that there can be no 

misinterpretation of data. 



Rank Order Coefficient of 
Correlation 

Ranlc order correlation or Spearman's "rho" is also 
commonly computed for survey data when different individuals 
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or groups rank order specific lists. For example, if groups 
were asked to rank order a set of items, the rankings could 
be analyzed to determine if they were similar or not. 
Table 16 presents a general model illustrating when "rho" 
could be used. 

Table 16, Model of Rank Ordered Items 



Group A Group B 

Rankings Variable Rankings Variable 

1 A 2 A 

2 B 3 B 

3 C 1 C 

4 D 5 D 

5 E 6 E 

6 F 4 F 



By computing "rho" it can be determined if a statis- 
tical difference exists between the rankings of the variable 
by the two groups. 

This technique is helpful when an investigator desires 
a ranking or preference for a set of items. Although 
statisticians allow for the ranking of up to 20 items in 
their formulae, in practice we have found that the ranking 
of more than eight or ten items is a source of confusion or 
frustration to a respondent. If several items, e.g., 16, are 
to be ranked; we suggest subdividing the items in several 
groups of four or eight so that multiple rankings could take 
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place* This technique eliminates the frustration of ranking 
large lists. One truly may not care about the eleventh, 
fourteenth, or fifteenth placed item* 

Again, by using a statistics textbook the formula 
and methods for "rho" can be found. 

The use of any statistical method is to determine 
the probability that chance played in creating the results. 
Statistics do not prove or verify conclusions. The appropri- 
ateness of the statistical treatment is more important than 
finding "a significant difference at the .01 level." 

The next chapter provides ideas on organizing and 
reporting survey results. 
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CHAPTER 9 



THE RESEARCH REPORT 

Planning the Report 

After conducting the survey, tabulating the data, 
and making appropriate conclusions, the next step is preparing 
a final written report. The preparation of such a report is 
essential since it is the means by which your results are 
disseminated. Further, most funding agencies require a 
final report. 

As briefly outlined below, a report plan should 
include an introduction and separate sections which present 
methods, results, and discussion. The four main parts of 
a report are described below. 

1. The Introduction states the goals of the study 
and describes previous related research. 

2. A Methods section describes the respondent 
population, the survey instrument, and the 
procedures used to conduct the study. 

3. The Results section reports the outcome of the 
analysis of data. 

4. A Discussion section interprets and explains the 
results, and identifies the limitations and short- 
comings of the study. 

A research report includes all relevant aspects of 
the study. At the same time, it is concise. The writer 
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must be organized, in order to clearly and accurately 
present the study in a manner that would allow the reader to 
repeat the procedure. 

Writing Style 

The writer should strive for objectivity, giving 
facts / rather than personal judgments. Simple words are 
preferred in the writing of the report. 

Word selection is an art. To convey the exact 
meanings and conclusions the writer must strive to select the 
words or concepts that carry the appropriate connotation and 
denotation. A few rules are listed below to help in writing 
a respectable survey report* 

Do not use trite phrases . "It is a proven fact that 
. . • ." "Everybody knows that . . . ." If you state a fact, 
present data to support it. In survey research, absolutely 
nothing is ever "proved." All results are tentative. The 
survey, if correctly conducted, simply reflects a relative 
mood of the respondents. In most cases, terms such as "it 
seems that . . ."or "it was apparently ..." are appropriate 
for survey reports. "Seems" and "apparently" connote a 
tentativeness about the findings. This is precisely what you 
desire to connote. 

Minimize personal references . Instead of "I believe 
. . . , write, "the data indicate . . . ." The reason for 
this emphasis is to demonstrate to the reader that the 
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generalizations and conclusions are based on data collected 
from an appropriate group* 

Be careful of Latinized singular and plural forms > 
Write these data , not this data. Data is the plural form; 
datum is singular. Criteria is plural; criterion is 
singular. You may think that this is being far too particular. 
No, it is simply knowing correct grammatical forms • Your 
report will be judged on how it is written, as well as what 
the findings are. 

Select the precise term . Do not substitute the word 
"percent" for "probability." Again, careful word choice is 
essential toward conveying the correct interpretation to the 
reader. A written report is strictly a one-way communication 
device. The reader cannot ask you to reinterpret a statement. 

Using "significant difference ." Do not write "the 

data are (or are not) significant." Instead, write, "The 

statistical analysis indicates there is a significant 

difference between . . . . " This example demonstrates how 

the written statement describes something. In this case the 

writer must denote that it was through statistical analysis 

that a significant difference is determined. Further, it is 

totally inappropriate to state that there is a "significant 

difference" when using data unless you determine statistical 

significance through appropriate parametric or nonparametric 

measures. If you do not compute such tests, then you must 

use ^some other description; such as "The forty percent 
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difference between the two groups in question 18 would 
indicate a very apparent discrepancy in opinion." Or, it 
could be written as "There was a wide range of responses, 
with a 40-point range being computed between the two groups." 
Do not write there were no results, when you mean that there 
was no significant difference between variables. Here again 
is demonstrated an often committed error in choosing a set 
of terms which convey an incorrect summarizing statement. 

Format . The format of a report should reflect 
careful organization in appropriate topics or themes. The 
sections of this chapter illustrate a sample format. As a 
matter of style, this entire guide has a very definite 
format. Have you observed the various headings? These act 
as an outline organizer and aid the reader. A few tips 
about format are listed below. 

Abbreviations . When using abbreviations there are a 
few general rules to which nearly all researchers subscribe. 
Use standard abbreviations. Write the word and the 
abbreviation together the first time the term appears in the 
report. In the remainder of the report just use the abbrevi- 
ation. Actually, a monogram is used. To illustrate this 
point consider the following: "After computing the grade 
point average (GPA) for the junior class the GPA was computed 
for all seniors." It is most important to always write out 
a word or set of words before abbreviating. This technique 

helps to keep the reader fully informed — not confused by 
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unknown monograms, acronyms, or abbreviations • Quotation 

marks should be used to abbreviate statistical tests, such 

as Spearman's "rho," Numbers less than 10 should be spelled 

out unless written in series, or as dates, scores, or units 

of measurement. 

Tense . To provide a rather uniform style and for 

consistency, the report should be generally written in one 

tense. Most reports are written in the past tense since the 

writers are describing some activity which took place. 

Write in the simple past tense, except when something 

is observed in the present. Use the future tense when 

speculating or addressing issues of the future. An important 

point to remember is that you are communicating to others . 

Use the general rules of writing. 

Write in the third person. The third person is 

preferred since it eliminates overuse of "I" or "We." 

Throughout this guide the writers (Note: third person form) 

have used third person construction. Other terms are the 

researcher or investigator. 

Source citation . All citations, quotes, or paraphrases 

must be identified for the reader. In this manner, the sources 

may be rechecked or examined by those who become more 

interested in the study. Avoid using footnotes; rather, cite 

the authors and the dates of relevant publications in the body 

of the paper and include in a bibliography. Observe how 

citations are accomplished in this guide. 
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All of the above mechanics of technical writing are 
the "nuts and bolts*" Let us now focus attention on the 
components of the report. 

Parts of the Research Report 

Introduction 

The introduction provides focus and establishes the 
context for the entire study. The introduction is analogous 
to a funnel shape: wide at the mouth and progressively 
narrowing to a smaller passageway. The opening statements 
provide a logical conceptual outline for the formulation of 
the remainder of the report. Begin the introduction by 
briefly summarizing the present state of knowledge. Next, 
present the process by which the current study evolved. 
Finally, pare the introduction down to the actual purpose and 
anticipated findings of the present study. 

Method 

The methods section provides detailed information 
about the manner in which the study was conducted. Provide 
enough information so that the survey could be replicated by 
a reader. At the same time, be concise, accurate, and 
precise. 

Respondent population . Describe the relevant 
characteristics of the population from which your sample was 
selected, e.g., age, sex, education, socioeconomic status, 
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and race. Explain the exact method used to select a sample 
from the population. 

Survey instrument . The questionnaire should be 
described in detail in this section. Discuss the subscales 
and identify the questions which compose each subscale. 
State the methods of tabulation and analysis of the data. A 
copy of the instrument should be included in an Appendix. 

Procedure . The procedure is a step-by-step 
description of your communications with the respondents, 
including the cover letter, instructions and follow-up 
procedures. This section also includes how data were treated, 
how the sample was selected, and any other details appropriate 
to the study. 

Results . 

The results section includes the quantitative values 
obtained from your data analyses. Graphs and tables are very 
helpful in summarizing data. For example, instead of listing 
the number of respondents who marked each category of an item, 
a frequency distribution table yields the same information in 
a simple, pictorial manner. 



Table 16. 


Example of 


Frequency 


Distribution Table 




Yes 


No 


Maybe 


Male 


15 


25 


5 


Female 


10 


20 


10 
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Tables can be constructed in many forms , illustrated by the 
various contingency tables presented in this manual. Select 
a layout for its simplicity and readability. 

Graphs and histograms are also recommended for data 
display. The bar diagram in Fig. 4 is frequently used to 
illustrate data. Note how the responses in Table 16 would 
appear in a histogram. 
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Fig. 4. — Example of Bar Diagram 



The bar diagram provides a visual picture of the frequency 
with which each category was marked, or the total response 
pattern of some question. 

The object of graphing is to present data accurately 
in an easy to understand form* Certain basic considerations 
are necessary. The axes of the graph are to be precisely 
labeled. The graph should be numbered as a "figure** and be 
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titled. It must be large enough to show clearly summary 
analysis. The area in which the graph is drawn should either 
be a square or a nearly rectangular square. Disproportionately 
tall or wide graphs should be avoided since they convey 
distorted images of the real situation, hence a visual bias. 
Finally, the axes of the graph should not be abridged, 
expanded, contracted, or distorted in such a way that they 
give a false impression of what the data indicate. 

Discussion 

The discussion section, in contrast to the introduction, 
is developed like an inverted funnel. It begins with the 
specific conclusions of the study and expands to discuss the 
general area of knowledge and the applicability of the results. 
In this section, first interpret and explain the results; then 
discuss any limitations of the study and conclusions. If 
appropriate, include a generalized statement of the population. 
Finally, summarize the present state of knowledge on which the 
study focuses. 

Be extremely careful to identify the (1) assumptions, 
(2 ) summaries , (3 ) generalizations , and (4 ) conclusions . 
Each of these four is very different. Assumptions and 
conclusions are usually subjective in characteristic. They 
are subject to the personal evaluation of the individual 
researcher. Suifcnaries simply give a short recapitulation of 
the survey resul^^s. Generalizations are those statements 
made from the summaries. 
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For example, if a study were conducted by a teacher 

who found that 80 percent of the senior boys intended to go 

to college after graduation while 38 percent of the senior 

girls intended to do so. The investigator might write the 

following, but very brief, discussion « 

Since the survey was administered without threat, 
the data from the seniors should be accurate and 
valid [the assumption] . The finding here is consistent 
with other studies reviewed by the investigator [a 
summary] . Therefore it appears that male seniors 
show a greater interest in going on to higher education 
after high school than do the female seniors at George 
High School. It might also be stated that male seniors 
have identified their post secondary school plans 
earlier than do female seniors [generalizations] . 

From these findings it seems that more vocational 
counseling is needed for the females so that post 
secondary school plans are formulated earlier in their 
careers [conclusion] . 

. It has been one of the writers of this guide's 

experience to observe reports where a writer was confused 

on all four concepts — and needless to say, the discussion was 

a disaster. 

Appendix 

You may wish to include an appendix at the end of 
your report to show all collected raw data, any detailed 
statistical treatments, or the survey instrument itself. The 
appendix should include only information that is unnecessary 
to be integrated in the body of the report. 

General 

The report should be typed or duplicated in a neat 

fashion. Use high quality photo reproductions or ink 
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duplicated copy. Ditto is usually unacceptable since it is 
so often dull and cannot be easily photocopied for further 
reproduction. 

Finally, consider writing a short article for your 
professional association journal. In this manner you 
communicate your results to your colleagues who share 
similar interests. Or, if the results are of local interest, 
then write a report for the school district newsletter. The 
final report is written to communicate. As the investigator, 
this is all your responsibility. 

In Conclusion 

This manual has been prepared to aid teachers in 
conducting better surveys. Our intent has been to provide 
not just a manual, but a total format. Throughout this book 
are examples that can be revised with little adaptation to 
fit almost any possible survey. We desire to impart the 
affective consequence that good surveys reflect a lot of hard 
thoughtful work. Good luck! 
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APPENDIX 



MODELS OF QUESTIONNAIRE FORMAT 

Within the Appendix are four different questionnaire 
formats, all of which are adapted from one survey instrument. 
The first model is the actual instrument which was mailed to 
obtain opinions and information from the vocational agricul- 
tural director in the State of Washington, public high schools, 
and high school principals. Of 311 instruments mailed, 301 
or 97% were returned. Following Model No. 1, other models 
will be illustrated to show how modifications may be made 
from a basic format. 

Model 1 was designed to accommodate ease of response. 
Note, double spacing, one variable per question, uncrowded 
appearance . 

Each question is key-coded to an electronic data 
processing card. Thus, question number one is punched in 
card column one. To aid the key punch operator, each row 
number is preceded. If a respondent checked that the major 
responsibility was a "Vocational Agriculture Teacher," the 
key punch operator would punch in column one, the row one 
punch. All items are coded for column and row designations. 
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Observe questions 3^ 7^ 8, 9, and 10. Here we 
followed the formula of balance. The items reflect all 
possible responses for the variables. 

Questions 12 and 16 required numerical information 
from the respondent. We predicted that the number would be 
0 through 9, and alloted one column for the quantification 
of those responses. 

Response categories were precoded by parentheses 
instruction, ( ) 1.1, where the respondent checked the space 
within the parentheses. 

Questions 1 and 2 will be altered in Models 2,3, 
and 4 to illustrate other features which may be incorporated 
into an instrument. 
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Model 1 



SURVEY OF VOCATIONAL AGRICULTURE 
TEACHER SUPPLY AND DEMAND 

Sponsored by 
The Office of Field Service and Research 
College of Education 
Washington State University 
Pullman, WA 99163 

DIRECTIONS 

Most items can be answered by placing a check (/ ) mark in the 
blank box ( ) . Please respond appropriately where information 
is requested for other questions. 

1. Please indicate your major responsibility by checking 
only one of the categories below. 



( 


) 


1. 


1 


Vocational Agriculture Teacher 


( 


) 


1. 


2 


High School Teacher 


( 


) 


1. 


3 


Administrator 


( 


) 


1. 


4 


Other (please specify) 



2. How many secondary school students attend your high 
school (grades 9-12 or 10-12) for 1974-75? 



( 


) 


2. 


1 


1,000 or more 


( 


) 


2. 


2 


750-999 


( 


) 


2. 


3 


500-749 


( 


) 


2. 


4 


250-499 


( 


) 


2. 


5 


249 or less 



IF NO VOCATIONAL AGRICULTURE (Vo-Ag) COURSES ARE BEING TAUGHT 
IN YOUR HIGH SCHOOL, PLEASE CONTINUE BELOW WITH QUESTION 3. 



IF THERE ARE VOCATIONAL AGRICULTURE (Vo-Ag) COURSES NOW BEING 
TAUGHT IN YOUR HIGH SCHOOL, PLEASE GO DIRECTLY TO QUESTION 7. 
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3* If there are no Vo~Ag courses or programs in your high 
school, are there plans for your high school to offer a 
Vo-Ag course or program? 

( ) 3.1 Yes 

( ) 3.2 No 

( ) 3.3 Do not know 



4. If a Vo-Ag course or program is planned for your high 
school, in which year will it start? 

'( ) 4.1 1975-76 school year 

( ) 4.2 1976-77 school year 

( ) 4.3 After 1977 school year 

( ) 4.4 Do not know 



5. & 6. If no Vo-Ag course or program is planned for the 
immedic>te future, please rank in order the first and 
second reasons which best describe why a course or 
program will not be added. Place a "1" in front of 
the primary reason. Place a "2" in front of the second 
most important reason. Please do not rank any other 
items. Just rank the first and second reasons, only. 

1 Not enough student interest 

2 No certified Vo-Ag teachers 

3 No need for agriculture training in our school 

4 Lack of facilities 

(5) (6) 

(1) (2) 5 Lack of money 

6 Levy failures 

7 Other subjects considered more important 

8 Other (please specify) ^ 



If you are not offering any Vo-Ag courses, you have completed 
your portion of this questionnaire. Please return this 
questionnaire in the enclosed envelope. Thank you for your 
cooperation. 
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Please check only one response for each question, 7 & 8* 
Please indicate how you perceive the teaching load of Vo--Ag 
teachers in relation to all other teachers for each of the 
following scales. 

7. ( ) 7.1 The teaching load in Vo-Ag is heavier than 
for most teachers in the high school. 

( ) 7.2 The teaching load in Vc-Ag is about equal to 
that of most teachers in the high school 

( ) 7.3 The teaching load in Vo~Ag is lighter than 
for most teachers in the high school. 



8. ( ) 8.1 The "conference or preparation" periods for 
Vo-Ag teachers are greater than for most 
teachers in the high school. 

( ) 8.2 The "conference or preparation" periods for 

Vo-Ag teachers are about equal to that of most 
teachers in the high school. 

( ) 8.3 The "conference or preparation" periods for 
Vo-Ag teachers are fewer than for most 
teachers in the high school. 



Questions 9 & 10 draw a distinction between classes and 
courses. Agriculture Science is an example of a course. The 
number of sections of Agriculture Science which are being 
taught equals the number of classes. 

9. Which of the following best compares this year's (197 4- 
75) Vo-Ag classes to next year's (1975-76) planned Vo-Ag 
classes ? 

( ) 9.1 Fewer classes will be offered in 1975-76. 
( ) 9.2 Same for both years. 

( ) 9.3 More classes will be offered in 1975-76. 

10. Which of the following best compares this year's (1974- 
75) Vo-Ag courses to next year's (1975-76) planned Vo-Ag 
courses ? 

( ) 10.1 Fewer courses will be offered in 1975-76. 
( ) 10.2 Same for both years. 

{ ) 10.3 More courses will be offered in 1975-76. 
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11, Next year's (1975-76) Vo--Ag curriculum will cause which 
of the following to happen to your staffing pattern? 



11.1 

11.2 

11.3 



No changes in staffing are anticipated. 

More certified Vo-Ag teacher (s) will be 
needed (expansion assumed) . 

Certified Vo-Ag teacher (s) will teach other 
subjects (reduction assumed). 



11.4 Certified Vo-Ag teacher (s) will be transferred 
to another school (reduction assumed). 

11.5 Other (please specify) 



12. If you checked 11.2, 11.3, 11.4, or 11.5 in question 11, 
approximately how many teachers will be involved? Place 
the number on the appropriate line . 

12.1 (11.2) More certified Vo-Ag teachers will 

be needed (expansion assumed) . 

12.2 (11.3) Certified Vo-Ag teacher (s) will 

teacher other subjects (reduction 
assumed) . 



12.3 (11.4) 



Certified Vo-Ag teacher (s) will be 
transferred to another school 
(reduction assumed). 



12.4 (11.5) Other (please specify) 



Questions 13, 14, and 15 concern some anticipated changes 
for 1975-76. You will only have to respond to one of the 
sets. Below is a quick summary. 



13. 
14. 
15. 



Respond to number 13 if no change in Vo-Ag classes is 
anticipated. 

Respond to number 14 if you anticipate increasing the 
number of Vo-Ag classes. 

Respond to number 15 if you anticipate decreasing the 
number of Vo-Ag classes. 
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13 ♦ If no change in the number of Vo-Ag classes ^ check only 
one statement that best describes your reason for the 
Vo-Ag curriculum plans for next year. 



) 13.1 We have the proper balance of classes offered 
at our school ♦ 

) 13.2 There is not enough student interest to 
warrant more classes. 

) 13.3 Not enough certified Vo-Ag teachers to enable 
a class increase. 

) 13.4 Vo-Ag Advisory Committee decision ♦ 

) 13 ♦ 5 Other (please specify) 



14 ♦ If increasing the number of Vo-Ag classes /> check only 

one statement that best describes your situation* 

) 14.1 Curriculum officials felt it advisable ♦ 

) 14.2 Student interest was high. 

) 14.3 School patrons felt it advisable. 

) 14.4 Vo-Ag Advisory Committee decision. 

) 14.5 Other (please specify) 



15 ♦ If decreasing the number of Vo-Ag classes / check only 

one statement that best describes your situation ♦ 

) 15.1 Curriculum officials felt it advisable. 

) 15.2 Student interest was low* 

) 15.3 School patrons felt it advisable. 

) 15.4 Vo-Ag teacher leaving the school ♦ 

) 15.5 Vo-Ag Advisory Committee decision. 

) 15.6 Other (please specify) 
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16. If Vo-Ag teachers are leaving your school during 1975 

for any of the following reasons, please place the number 
of those leaving on the appropriate line(s). 



Number 
Leaving 





Reason for 
Leaving 




16 


Do not know 




17 


Retiring 




18 


Moving to administration 




19 


Moving to Community College 




20 


Teaching in another state 




21 


Switching to farming 




22 


Switching to agri-business 




23 


Switching to non-agriculture 


business 


24 


Switching to non-agriculture 


teaching 


25 


Serving in military 




26 


Deceased 




27 


Attending graduate studies 




28 


Other (please specify) 





Thank you for your cooperation. Please return this instrument 

in the stamped, self -addressed envelope which is enclosed. 

If you have any questions concerning this study, kindly contact : 

Dr. Donald C. Orlich 
Professor of Education 
Cleveland Hall 
Washington State University 
Pullman, WA 99163 

NOTE: A report of this survey is to be presented at the annual 
Washington Vocational Agriculture Teachers Association 
(WVATA) in Spokane, July 21-25, 1975. 

For survey use only ID 

29 30 31 
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Model 2 is an adaptation of the original in that the 
responses use a short blank space and only one number, i.e., 
the row number preceding each response is preprinted. 
Parentheses or square brackets could be substituted for the 
blank space without changing the basic format. 
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Model 2 



SURVEY OF VOCATIONAL AGRICULTURE 
TEACHER SUPPLY AND DEMAND 

Sponsored by 
The Office of Field Service and Research 
College of Education 
Washington State University 
Pullman, WA 99163 

DIRECTIONS 

Most items can be answered by placing a check (/ ) in the 
blank box ( ) . Please respond appropriately where information 
is requested for other questions. 

1. Please indicate your major responsibility by checking 
only one of the categories below. 

1 Vocational Agriculture Teacher 

2 High School Teacher 

3 Administrator 



4 Other (please specify) 



2. How many secondary school students attend your high 
school (grades 9-12 or 10-12) for 1974-75? 

1 1/000 or more 



2 750-999 



3 500-749 



4 250-499 



5 249 or less 



IF NO VOCATIONAL AGRICULTURE (Vo-Ag) COURSES ARE BEING TAUGHT 
IN YOUR HIGH SCHOOL, PLEASE CONTINUE BELOW WITH QUESTION 3. 

IF THERE ARE VOCATIONAL AGRICULTURE (Vo-Ag) COURSES NOW BEING 
TAUGHT IN YOUR HIGH SCHOOL, PLEASE GO DIRECTLY TO QUESTION 7. 
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Model 3 illustrates a feature of having each 
respondent code the questionnaire for tabulating or key 
punching. The respondent places the forced response code 
item in a designated area. The placement of the code item 
helps reduce some work for the researcher. Using this 
technique, the respondent places the key punch row number in 
the blank numbered space. The numbered space indicates the 
question number as well as the EDP card column number. The 
technique illustrated in Model 3 thus reduces the chances of 
transposition errors by the researcher or key punch operator. 
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SURVEY OF VOCATIONAL AGRICULTURE 
TEACHER SUPPLY AND DEMAND 

Sponsored by 
The Office of Field Service and Research 
College of Education 
Washington State University 
Pullman, WA 99163 

DIRECTIONS 

MOST ITEMS CAN BE ANSWERED BY INDICATING IN THE 
COLUMN AT THE RIGHT YOUR APPROPRIATE CODE RESPONSE. 
PLEASE RESPOND APPROPRIATELY WHERE INFORMATION IS 
REQUESTED FOR OTHER QUESTIONS. 

1. Your major responsibility is: 

1 Vocational Agriculture Teacher 

2 High School Teacher 

3 Administrator 

4 Other (please specify) 



2. How many secondary school students attend 
your high school (grades 9-12 or 10-12) for 
1974-75? 



1 


1,000 or more 


2 


750-999 


3 


500-749 


4 


250-499 


5 


249 or less 



IF NO VOCATIONAL AGRICULTURE (Vo-Ag) COURSES ARE 
BEING TAUGHT IN YOUR HIGH SCHOOL, PLEASE CONTINUE 
BELOW WITH QUESTION 3. 



IF THERE ARE VOCATIONAL AGRICULTURE (Vo-Ag) COURSES 
NOW BEING TAUGHT IN YOUR HIGH SCHOOL, PLEASE GO 
DIRECTLY TO QUESTION 7. 
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Model 4 is an adaptation of Model 3. The respondent 
simply places a check in the appropriate parentheses. The 
investigator then by hand, places the coding information in 
the right-hand two coliimns. This allows the investigator to 
make all decisions on the responses. The key punch operator 
would simply follow the information recorded in the right 
hand coding area. 
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Model 4 



SURVEY OF VOCATIONAL AGRICULTURE 
TEACHER SUPPLY AND DEMAND 

Sponsored by 
The Office of Field Service and Research 
College of Education 
Washington State University 

Pullman, WA 99163 PLEASE 

DO NOT MARK IN 
DIRECTIONS THIS AREA 



Most items can be answered by placing a 
check (/ ) mark in the blank box ( ) . Please 
respond appropriately where information is 
requested for other questions. 



1- 



Please indicate your major responsi- 
bility by checking only one of the 
categories below. 

( ) Vocational Agriculture Teacher 

( ) High School Teacher 

( ) Administrator 

( ) Other (please specify) 



2. 



How many secondary school students 
attend your high school (grades 9-12 
or 10-12) for 1974-75? 



) 1,000 or more 

) 750-999 

) 500-749 

) 250-499 

) 249 or less 



IF NO VOCATIONAL AGRICULTURE (Vo-Ag) COURSES 
ARE BEING TAUGHT IN YOUR HIGH SCHOOL, PLEASE 
CONTINUE BELOW WITH QUESTION 3. 

IF THERE ARE VOCATIONAL AGRICULTURE (Vo-Ag) 
COURSES NOW BEING TAUGHT IN YOUR HIGH SCHOOL, 
PLEASE GO DIRECTLY TO QUESTION 1. 
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